-- Logs begin at Mon 2022-08-08 12:52:07 UTC, end at Mon 2022-08-08 14:03:18 UTC. -- Aug 08 13:20:35.706086 dv-cinder-851996-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 08 13:20:35.715676 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Aug 8 13:20:35 2022] *** Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: nodename: dv-cinder-851996-6-iscsi Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: machine: x86_64 Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: clock source: unix Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: pcre jit disabled Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: detected number of CPU cores: 4 Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: current working directory: / Aug 08 13:20:35.717727 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: detected binary path: /usr/bin/uwsgi-core Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: your processes number limit is 47680 Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: your memory page size is 4096 bytes Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: detected max file descriptor number: 2048 Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: lock engine: pthread robust mutexes Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: thunder lock: enabled Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 08 13:20:35.718285 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 08 13:20:35.757716 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: Python main interpreter initialized at 0x55f56387b9c0 Aug 08 13:20:35.757716 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: python threads support enabled Aug 08 13:20:35.757716 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: your server socket listen backlog is limited to 100 connections Aug 08 13:20:35.757716 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: your mercy for graceful operations on workers is 90 seconds Aug 08 13:20:35.758174 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: mapped 403077 bytes (393 KB) for 2 cores Aug 08 13:20:35.758174 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: *** Operational MODE: preforking *** Aug 08 13:20:35.758459 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: *** uWSGI is running in multiple interpreter mode *** Aug 08 13:20:35.758459 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: spawned uWSGI master process (pid: 80177) Aug 08 13:20:35.758685 dv-cinder-851996-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 08 13:20:35.763126 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: spawned uWSGI worker 1 (pid: 80187, cores: 1) Aug 08 13:20:35.763126 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: spawned uWSGI worker 2 (pid: 80188, cores: 1) Aug 08 13:20:35.763126 dv-cinder-851996-6-iscsi devstack@keystone.service[80177]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 08 13:20:41.200805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 13:20:41.200805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: from cryptography.utils import int_from_bytes Aug 08 13:20:41.201648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 13:20:41.201648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: from cryptography.utils import int_from_bytes Aug 08 13:20:41.208537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 13:20:41.208537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: from cryptography.utils import int_from_bytes Aug 08 13:20:41.209066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 08 13:20:41.209066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: from cryptography.utils import int_from_bytes Aug 08 13:20:41.250700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 13:20:41.250913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 13:20:41.251021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] command line args: [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 13:20:41.251366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 13:20:41.251547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ================================================================================ {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 13:20:41.251870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] admin_token = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.252176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] config_dir = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.252398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.252592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] config_source = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.252810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] debug = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.253006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_config_append = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.254889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.255069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_dir = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.255324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_file = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.255595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.255903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.256213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.256493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.256672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.256844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.257116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.257394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.257671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.257957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.258225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.258361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 08 13:20:41.258511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.258598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 08 13:20:41.258745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] command line args: [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 08 13:20:41.258831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.259061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 08 13:20:41.259178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.259267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ================================================================================ {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 08 13:20:41.259508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] admin_token = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.259596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.259798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] config_dir = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.259987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] config_source = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] publish_errors = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] debug = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.260994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.261983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_config_append = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] standard_threads = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_dir = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_file = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.262977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] use_journal = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] use_json = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] use_stderr = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] use_syslog = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.263977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.264190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.264283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.264438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.264524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.264691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.264833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.264916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.265117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.265205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.265345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.265426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.265590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.265753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.265902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.266062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.266142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.266423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.266582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.33/identity {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.266766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] publish_errors = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.266845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.266997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.267255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.267335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.267421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.267623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.267703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.267880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.267960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.268115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] standard_threads = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.268211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.268345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.268424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.268579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.268657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.268750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.268836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.268915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] use_journal = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.269055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.269136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] use_json = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.269242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] use_stderr = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.269319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.269403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] use_syslog = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.269529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.269615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 08 13:20:41.269779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.269859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.270861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.271967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.272922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.273941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.274951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.275889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.276973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.277982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.278899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.279964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.280974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.281873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.282960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283161 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.283899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.284895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.285978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.286999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.287926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.288990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.289949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.external = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.290924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.password = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] auth.token = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.291945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.292861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.293962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.external = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.294936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295191 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.password = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] auth.token = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.295927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.296936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.297875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.298999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.299974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.300927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.301975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.302848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.303994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304073 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.304934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.305985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.306924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.307899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.308968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.309998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.310928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.311865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.312956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.313892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.314874 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.315607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.316962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.317897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.318974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.319932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.320954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.321943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] role.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.322871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] role.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.323854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.324960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.325916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.326894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] role.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] role.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.327946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.328964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.329896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.330877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.331977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.332925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.333895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.334995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.connection = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.335969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.336953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.337975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.338909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.339863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.340022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.340111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.340187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80187) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 13:20:41.340332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.connection = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.340586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.340834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.341072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.341312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.341550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.341785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.342020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.342256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.342501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.342737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.342980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.343239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.343490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.343726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.343971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.344203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.344443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.344681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 08 13:20:41.344849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80188) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 08 13:20:41.369684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.369945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.370136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.370863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.371164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.371320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.380217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.381446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.409956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.410151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.410401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.410633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.462604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.462830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.513972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.514166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.514239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.514331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.579935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.580053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.580167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.580248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.580492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.603546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.603745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.603899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.604950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.605176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.616074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.616439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.628906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.629936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.630062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.630132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80187) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.630196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.630353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.630566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80188) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 08 13:20:41.647341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.647497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 13:20:41.647889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 13:20:41.648162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 13:20:41.648302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 08 13:20:41.648800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.648876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.649089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.649213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.649364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.649710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.660606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.660895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.661156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.661282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.661439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.661589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.661860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.662148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.668495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.669214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.671630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.671912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.672183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.672310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.672486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.672614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.672775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.672901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.673993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.674968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.697804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.698258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.699318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.699616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.699746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.700068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.704242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.704531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.704733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.704873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.705026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.705299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.711411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.711669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.712133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.712433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.715859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.716095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.716379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.716549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.716650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.716863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.716990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.717990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.718267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.718565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.733526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.733779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.734620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.734912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.737922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.738942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.741060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.741356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.742147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.742451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.745912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.747003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.749118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.750214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.752285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.752582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.752869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.753981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.754298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.754582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.754876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.770412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.770709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.770978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.771266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.771580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.771887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.772162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.772433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.778205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.779574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.779776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.780022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.781166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.781460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.782856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.783129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.783449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.783739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.784022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.784324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.784614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.784916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.785223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.785507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.795564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.795906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.796109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.796418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.796696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.796970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.797093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.797215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.797387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.797671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.797956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.798240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.798499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.798800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.814178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.815773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.817194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.818732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.820145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.821743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.821866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.823259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.823383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.823521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.823834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.824870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.825173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.825449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.830271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.831914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.832038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.833574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.834846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.835139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.835447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.835726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.836569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.836862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.837202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.837482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.843913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.844220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.844576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.845688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.845987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.846270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.851497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80187) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.851784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.852072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.852345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.852646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.852919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.853194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.853328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80188) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 08 13:20:41.853447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.853581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.853832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.853918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.854922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.855009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.855217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80187) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.855305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.855604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.855899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.856176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.856445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.856726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.856990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80188) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 08 13:20:41.888964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f56387b9c0 pid: 80187 (default app) Aug 08 13:20:41.890985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f56387b9c0 pid: 80188 (default app) Aug 08 13:20:41.892425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44047063-10b8-4459-a397-329cb2dbc058 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:41.892708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44047063-10b8-4459-a397-329cb2dbc058 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:41.892977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44047063-10b8-4459-a397-329cb2dbc058 None None] PATH_INFO: `/v3` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:41.897315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1/1] 10.222.0.33 () {50 vars in 791 bytes} [Mon Aug 8 13:20:41 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:20:49.498524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:49.499168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:49.499168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:49.503555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1/2] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:49.509666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:49.510522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:49.510522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:49.583506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: INFO dbcounter [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] Registered counter for database keystone Aug 08 13:20:49.604794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG oslo_db.sqlalchemy.engines [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80187) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 13:20:49.611940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writer thread running {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 13:20:49.790461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG passlib.handlers.bcrypt [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80187) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 13:20:49.790885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG passlib.handlers.bcrypt [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80187) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 13:20:49.845643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:49.847733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:49.985218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 2/3] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 13:20:50.140845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:50.211102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: INFO dbcounter [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] Registered counter for database keystone Aug 08 13:20:50.242272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG oslo_db.sqlalchemy.engines [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80188) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 08 13:20:50.250905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writer thread running {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 08 13:20:50.370820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee942a96-02f2-475e-a8a5-fff718dddd5d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:50.375597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:50.376256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:50.376536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:50.376805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:50.386285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:50.437543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 13:20:50.437885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b40a16f-5eba-42ca-a263-892c623e4459 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:50.448305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 2/4] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Aug 8 13:20:50 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:20:50.454571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:50.469529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b4f7ef-97ed-46ba-af60-19e8aeef8b7d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:50.475212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:50.476335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:50.476615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:50.476888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] PATH_INFO: `/v3/projects/cd8972f6a1684e42be6181a89144c88c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:50.478444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] RBAC: Authorizing `identity:get_project(project_id=cd8972f6a1684e42be6181a89144c88c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:50.528799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 08 13:20:50.529405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631ea21-14d4-4041-98b8-89b9bfabdfcc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:50.533884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 3/5] 10.222.0.33 () {56 vars in 1207 bytes} [Mon Aug 8 13:20:50 2022] GET /identity/v3/projects/cd8972f6a1684e42be6181a89144c88c => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:20:52.516893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8fe0-f9c3-41b6-bff9-3d7d441e0160 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:52.517442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8fe0-f9c3-41b6-bff9-3d7d441e0160 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:52.517442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd8fe0-f9c3-41b6-bff9-3d7d441e0160 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:52.521625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 3/6] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:52.527387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:52.527816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:52.527935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:52.551308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:52.552863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:52.561978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 4/7] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 08 13:20:52.584605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd8fe0-f9c3-41b6-bff9-3d7d441e0160 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:52.598964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd8fe0-f9c3-41b6-bff9-3d7d441e0160 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:52.604287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:52.604912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:52.605186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:52.605457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:52.613716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:52.615572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f1cc9-1ecd-4a59-8af4-2cfcf15cb368 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:52.624710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 4/8] 10.222.0.33 () {58 vars in 1324 bytes} [Mon Aug 8 13:20:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:20:52.631065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:52.641184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44511ccb-8b97-4768-8d5b-75030f7c247e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:52.646738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:52.647421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:52.647702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:52.647969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:52.649476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:52.651392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0ce1b-fc1b-4677-979c-c70a77e77ff4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:52.653012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 5/9] 10.222.0.33 () {56 vars in 1198 bytes} [Mon Aug 8 13:20:52 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:20:54.710113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750779a-4963-4795-86d0-18116e75f07f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:54.710690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750779a-4963-4795-86d0-18116e75f07f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:54.710690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8750779a-4963-4795-86d0-18116e75f07f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:54.712049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 5/10] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:54.717518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:54.717997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:54.718097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:54.736637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:54.737797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:54.739501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8406-577d-45fb-9e79-e30504ff8213 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:54.739973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8406-577d-45fb-9e79-e30504ff8213 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:54.740248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8406-577d-45fb-9e79-e30504ff8213 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:54.741607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 6/11] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:54.746765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:54.747048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:54.747349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:54.790740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG passlib.handlers.bcrypt [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80188) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 08 13:20:54.791196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG passlib.handlers.bcrypt [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80188) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 08 13:20:54.820346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 6/12] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:54.836180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:54.837930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:54.843609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:54.867891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:54.937290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 7/13] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:54.948584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088585e-79f4-4284-8c57-e488d1d13cc9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:54.959373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:54.982816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:54.983412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:54.983691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:54.983965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:54.995439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:20:55.004141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:20:55.005085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:20:55.005911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 7/14] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:20:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:20:55.011647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.063471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3705ba6b-3b5a-4c29-82cb-bc25865790b8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.065930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.095314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.101179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.102054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.102329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.102606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.114659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2c46a6-c66b-4e80-9ab9-da5514de7cf1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.150317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.150913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.151213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.151501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.152151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:20:55.152471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.153889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.165777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 8/15] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:55.170546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.171244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.172126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.280130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 8/16] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:20:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:20:55.285787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.288953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.390137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e323522b-2b5d-4eb9-9d35-8444a6f16147 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.402892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc75cfb-84f1-4a26-8df0-cb62b89a6bb2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.425778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.426648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.426928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.427240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.445738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.446570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.447490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.447700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.447910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.448320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.449895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.451747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.453337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 9/17] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:55.458957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:20:55.465681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:20:55.466558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:20:55.467382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:20:55.467382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:20:55.467382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:20:55.467382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 9/18] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:20:55.474290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.475052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.531626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.560049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a464aabc-1e79-47d9-984c-7c23e0e72535 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.577122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c23b36-de93-40e3-b848-9309111ec463 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.593133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.593721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.594002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.594271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.595428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.596932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.598499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 10/19] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:55.604437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:55.614005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.614629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.614920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.615230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.615889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:20:55.616195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.617626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f826004f-64b9-4b4f-ae3c-0c384b8ab67b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.635556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 10/20] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:55.694674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:55.718687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0e1873-b8b7-4ef1-a9fb-0f04dd29a407 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:55.757115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:55.757749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:55.758028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:55.758297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:55.759231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.domain.id=default, user.id=69e577935e2a4950900c965d118c5e00) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:20:55.759623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:55.761768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a63347-ae0a-47d1-9e35-1ed68c0e130d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:55.769601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 11/21] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Aug 8 13:20:55 2022] GET /identity/v3/role_assignments?user.id=69e577935e2a4950900c965d118c5e00&scope.domain.id=default&role.id=193a5d91676042449492edcffd7d1760 => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:58.084713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946df35-6e3e-4365-88e3-be256f71b1e6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.085246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946df35-6e3e-4365-88e3-be256f71b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.085246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946df35-6e3e-4365-88e3-be256f71b1e6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.087359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 11/22] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:58.093430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.093961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.094163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.124866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:58.126105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:58.238528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22535960-dfba-4dc2-8a30-f593498d1457 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.238875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22535960-dfba-4dc2-8a30-f593498d1457 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.239093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22535960-dfba-4dc2-8a30-f593498d1457 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.243215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 12/23] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:58.244110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 12/24] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:58.250159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757cef-b908-47fc-9131-8394e72db73b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.250443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757cef-b908-47fc-9131-8394e72db73b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.250721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1757cef-b908-47fc-9131-8394e72db73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.280384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a1757cef-b908-47fc-9131-8394e72db73b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:58.281650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a1757cef-b908-47fc-9131-8394e72db73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:58.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:58.431788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:58.435466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 13/25] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:58.437909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c64b-1153-4d1d-8bbf-e121d2a45f40 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.438427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c64b-1153-4d1d-8bbf-e121d2a45f40 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.438920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3c64b-1153-4d1d-8bbf-e121d2a45f40 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.442437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 14/26] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:58.448421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.450896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.454381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.475818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc35aa5-e65b-47dc-bc26-4b9318d11ed3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:58.485306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:58.486546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:58.541043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:58.541769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.542053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.542328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.556179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:58.561790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:58.563593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:58.593923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 15/27] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:20:58.599044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:20:58.604322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:20:58.605261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:20:58.606065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:20:58.606065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:20:58.606065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:20:58.616511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 13/28] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:20:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:20:58.623096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:58.771052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bdef3-6c10-496b-8dd2-ba9732b81bc3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:58.775361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c640ac3-3414-4384-baa9-96aa458d25e5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:58.802042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:58.812369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:58.812812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:58.813243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.813620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.813888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.814057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.814455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.814872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.814984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:20:58.815375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:58.815974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:58.816980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:58.817678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0c195e-f52f-446e-aec7-f7de1a0d2d9b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:58.837866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 14/29] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:20:58.838707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:58.954784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983a50b-f399-470b-a411-ee45178c6047 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:58.994334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:58.995191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:58.995597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:58.995961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:58.996731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:58.998424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9bef8-8a67-421e-8e4f-218d3bfa5f84 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:59.080639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 16/30] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 13:20:58 2022] POST /identity/v3/roles => generated 219 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:20:59.082625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97f20-7504-4db7-a7d5-a483420769e8 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.083212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97f20-7504-4db7-a7d5-a483420769e8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.083730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e97f20-7504-4db7-a7d5-a483420769e8 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.085317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 17/31] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:59.086819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97f20-7504-4db7-a7d5-a483420769e8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.128683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 15/32] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 13:20:58 2022] POST /identity/v3/roles => generated 225 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:20:59.136690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.137553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954769e-92cb-4bf7-acca-95ea78422f02 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.137849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954769e-92cb-4bf7-acca-95ea78422f02 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.138245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954769e-92cb-4bf7-acca-95ea78422f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.188524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d954769e-92cb-4bf7-acca-95ea78422f02 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:59.190258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d954769e-92cb-4bf7-acca-95ea78422f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:59.227758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e97f20-7504-4db7-a7d5-a483420769e8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.271433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:59.272487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.273065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.273558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.274779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:59.277015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.278298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:59.327430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 18/33] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/projects => generated 301 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:20:59.334064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 16/34] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:59.370206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.470708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103b39-203a-44e6-8264-8885e19732c5 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.471380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103b39-203a-44e6-8264-8885e19732c5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.471933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103b39-203a-44e6-8264-8885e19732c5 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.473606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 17/35] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:20:59.481362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.481975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.482457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.504107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.509778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:59.511862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:59.529236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bfabf-942b-41e7-97eb-88ec34e5f8ad None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.578695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:59.579428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.579709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.580075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.595704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:20:59.604628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:20:59.605516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:20:59.606272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:20:59.609828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 19/36] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:20:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:20:59.615705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.617671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 18/37] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:59.640988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.721045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded82fc2-c123-46f6-a85c-cac2c33a5593 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.740619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39360f9c-f2ad-45d7-9a9a-caf625938681 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.748502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.755056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:59.755702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.755974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.756249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.756986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:20:59.757292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:59.758704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb290fef-adca-414d-8f9f-15d1ce62ff59 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:59.771564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 20/38] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:20:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:20:59.772894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c126ab-9c95-44e2-96a1-bc6766a6a90f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.773180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c126ab-9c95-44e2-96a1-bc6766a6a90f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.773452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c126ab-9c95-44e2-96a1-bc6766a6a90f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.779024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 21/39] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:20:59.779707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c126ab-9c95-44e2-96a1-bc6766a6a90f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.782215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:59.782957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.783265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.783544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.784681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:59.786305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be589191-27c3-4e48-b65a-6a15b523e0bb None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:59.787958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 19/40] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:20:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:59.789067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.789360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.789628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.793177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.807976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:20:59.809180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:20:59.875376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 20/41] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:20:59.876138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.880628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c126ab-9c95-44e2-96a1-bc6766a6a90f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.914918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:20:59.915610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:20:59.915889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:20:59.916158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:20:59.917521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:20:59.918985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:20:59.920595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 22/42] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:20:59 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:20:59.921338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:20:59.984715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa591a-d43e-4fab-b771-469a01ee4bf6 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:20:59.987282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:20:59.997138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:00.019164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124dd77-fc03-4f9b-aa9a-c4ed3712bf50 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:00.019619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:00.020242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:00.020528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:00.020796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:00.021443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:00.022891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:00.052915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 21/43] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:00.053101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:00.053612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:00.053895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:00.054083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:00.054260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:00.054926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:00.056540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:00.074397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 23/44] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 13:20:59 2022] POST /identity/v3/roles => generated 223 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:00.194485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f84275-f202-4b49-82f6-0bc5e788d610 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:00.237520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:00.238382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:00.238755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:00.239181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:00.240431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:00.241954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83ff080-19ec-443f-aeac-ecc880238411 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:00.243731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 22/45] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:00.249588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:00.356854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24651f37-d95c-49c4-8e5b-dced6b98e2f2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:00.395101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:00.395774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:00.396052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:00.396324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] PATH_INFO: `/v3/domains/default/users/69e577935e2a4950900c965d118c5e00/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:00.398327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=69e577935e2a4950900c965d118c5e00, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:00.399844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d09c18-7585-4272-acc3-82deaf4409ca None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:00.411307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:00.416752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 24/46] 10.222.0.33 () {58 vars in 1361 bytes} [Mon Aug 8 13:21:00 2022] PUT /identity/v3/domains/default/users/69e577935e2a4950900c965d118c5e00/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:02.081358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde53bf-f42a-4930-88e1-47e53a7b0511 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.081960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde53bf-f42a-4930-88e1-47e53a7b0511 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.081960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfde53bf-f42a-4930-88e1-47e53a7b0511 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.083378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 23/47] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:02.089045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.089499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.089675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.108558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:02.110075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:02.179120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 25/48] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:02.202101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde53bf-f42a-4930-88e1-47e53a7b0511 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:02.228778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:02.307202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfde53bf-f42a-4930-88e1-47e53a7b0511 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:02.341615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:02.342170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.342443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.342721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.343862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:02.345302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0bace6-d7b9-4b80-ab3f-2069f017336a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:02.346868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 24/49] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:02.352842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:02.426249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5bcc-7ebc-4927-b17c-22140a9a5dc3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.426547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5bcc-7ebc-4927-b17c-22140a9a5dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.426822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d5bcc-7ebc-4927-b17c-22140a9a5dc3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.428236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 25/50] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:02.433254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.433576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.433853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.452657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:02.453887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:02.457609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131053bf-d98c-48e7-a922-630c549bb622 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:02.485920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:02.494088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:02.494707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.494988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.495297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.495949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:02.497441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:02.521803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 26/51] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:02.535108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 26/52] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Aug 8 13:21:02 2022] POST /identity/v3/projects => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:21:02.545958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:02.572596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:02.656648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f7e23-00a2-4500-b20f-e6b15c04be0c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:02.696800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:02.697456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.697742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.698014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.708609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:02.713501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:02.714377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:02.715158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 27/53] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:02.720737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:02.821189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d2462-c425-4443-9fab-fe80df009f6f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:02.857311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:02.857935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:02.858220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:02.858498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:02.859217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:02.859527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:02.860926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:02.870208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 27/54] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:02.875886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:02.960968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:02.980311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4236c-567f-4f06-b32e-db8f3d2eee77 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:03.016263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:03.016914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:03.017196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:03.017472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:03.018625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:03.020134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:03.021759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 28/55] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:02 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:03.027585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:03.072145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:03.128695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2f12cf-ac43-4686-a9cf-67bc12fc1aa8 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:03.162822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:03.163513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:03.163793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:03.164067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:03.165311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:03.166773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606acb5c-a3f6-4a30-948f-b1e4b46a8573 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:03.168404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 28/56] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:03.175747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:03.280330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c59d0a-9387-4f3f-b3d0-0f41f68d5784 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:03.314906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:03.315522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:03.315809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:03.316084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:03.316815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.domain.id=default, user.id=69e577935e2a4950900c965d118c5e00) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:03.317128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:03.318583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c996e21-844c-4047-85d1-c9714235d32c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:03.324500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 29/57] 10.222.0.33 () {56 vars in 1322 bytes} [Mon Aug 8 13:21:03 2022] GET /identity/v3/role_assignments?user.id=69e577935e2a4950900c965d118c5e00&scope.domain.id=default&role.id=193a5d91676042449492edcffd7d1760 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:04.524378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d6f-58b8-400f-8fc6-17e6501e73b0 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:04.525001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d6f-58b8-400f-8fc6-17e6501e73b0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:04.525001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d6f-58b8-400f-8fc6-17e6501e73b0 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:04.526364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 29/58] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:04.531634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:04.532050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:04.532189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:04.551389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:04.552614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:04.562204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:04.624767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 30/59] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:04.649083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1d6f-58b8-400f-8fc6-17e6501e73b0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:04.754193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1d6f-58b8-400f-8fc6-17e6501e73b0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:04.765469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:04.790782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:04.791454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:04.791737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:04.792012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:04.793186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:04.794685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478484f-ba3b-4f42-9e79-f8ee4f023cc1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:04.796323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 30/60] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:04.802426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:04.910446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5044b508-9545-44f7-99a9-c6134fc055d9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:04.945989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:04.946630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:04.946910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:04.947220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:04.947871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:04.949337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d89d5d-faf7-4614-9860-44ae1a26065e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:04.973658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:04.978711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 31/61] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:04 2022] POST /identity/v3/users => generated 275 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:08.370839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b1618-453b-4788-b6de-dc087b8f6e05 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.372367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b1618-453b-4788-b6de-dc087b8f6e05 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.372881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823b1618-453b-4788-b6de-dc087b8f6e05 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.375859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 31/62] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:08.380160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ac7ee-485f-4766-8a16-e208f7bc9cae None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.381101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ac7ee-485f-4766-8a16-e208f7bc9cae None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.381575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ac7ee-485f-4766-8a16-e208f7bc9cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.415982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5e7ac7ee-485f-4766-8a16-e208f7bc9cae None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:08.417323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5e7ac7ee-485f-4766-8a16-e208f7bc9cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:08.466090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a571-45fa-49f1-a4b2-5e3ccc26f56a None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.466422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a571-45fa-49f1-a4b2-5e3ccc26f56a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.466650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100a571-45fa-49f1-a4b2-5e3ccc26f56a None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.468422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 32/63] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:08.473533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.473818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.474103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.500124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:08.501349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:08.550551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 32/64] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:08.552684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6912-98a3-4d8f-95f8-2e60ac860a9f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.553156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6912-98a3-4d8f-95f8-2e60ac860a9f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.553754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b6912-98a3-4d8f-95f8-2e60ac860a9f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.555675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 33/65] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:08.569557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.570021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.570471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.601126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:08.603278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:08.614442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 33/66] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:08.623202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:08.747222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 34/67] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:08.749517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:08.816123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ac32b-175a-4f52-ba52-bd7d909bb8a7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:08.839127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:08.875458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:08.884707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:08.886084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.886572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.887031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.906433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:08.915987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:08.916986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:08.917738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 34/68] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:08.918928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:08.944769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5161889-043a-4bad-88d4-da64daf417f2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:08.981256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:08.982075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:08.982530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:08.982869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:08.996665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-39269395-5e56-4461-95f9-7ab1adcac137 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:09.002199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:09.003045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:09.003871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 35/69] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:09.006573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780c0b0-9f72-4e47-8592-e6e26ea2ca3d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.006871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780c0b0-9f72-4e47-8592-e6e26ea2ca3d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.007197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780c0b0-9f72-4e47-8592-e6e26ea2ca3d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.008822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 36/70] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:09.009569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780c0b0-9f72-4e47-8592-e6e26ea2ca3d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.036890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75714e8e-48d3-4060-b2c8-22c45ac47ce9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.073082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.073734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.074010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.074283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.085339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1c88da30-a597-47e4-aafa-675d780a0446 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:09.090441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:09.092577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 35/71] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:09.094186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b030-5010-4db0-96c8-1178643addb0 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.095614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b030-5010-4db0-96c8-1178643addb0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.095614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3806b030-5010-4db0-96c8-1178643addb0 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.095614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 36/72] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:09.095928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b030-5010-4db0-96c8-1178643addb0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.122651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780c0b0-9f72-4e47-8592-e6e26ea2ca3d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.125758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.144118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.163707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.164339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.164623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.164893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.165738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:09.166046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.167538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809cfe1a-2e0d-4707-8956-38d6ae202ba8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.176690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 37/73] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.177679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bf16-6363-4cdc-81f2-f45c5b20dafb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.177969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bf16-6363-4cdc-81f2-f45c5b20dafb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.178245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b3bf16-6363-4cdc-81f2-f45c5b20dafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.197266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-34b3bf16-6363-4cdc-81f2-f45c5b20dafb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:09.198524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-34b3bf16-6363-4cdc-81f2-f45c5b20dafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:09.215347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3806b030-5010-4db0-96c8-1178643addb0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.252332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.252888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.253174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.253517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.254309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:09.254672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.256293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.265883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 37/74] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.268449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.273772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 38/75] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:09.275598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.276060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.276506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.304199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:09.306045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:09.395979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 39/76] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:09.397021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.409532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.421824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610ee135-26e8-4bfc-ae21-1f2068a47e90 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.444976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.458767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.459428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.459710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.459981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.460680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:09.460977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.462516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.472234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 38/77] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.473108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.521279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273837d4-358d-4454-80fd-ecc3742725ff None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.558491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.559169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.559450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.559713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.574083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.575651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.577395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 40/78] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.578207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.596581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6307f0-786c-477a-9be5-017a8eaff279 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.632165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.632809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.633095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.633368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.634646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.636581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.638162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 39/79] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.638947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.662433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.685592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.689253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db262a18-ee0f-4ada-acb4-dcc86ad78d28 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.725555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.726218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.726500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.726769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.727988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.729467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.731096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 41/80] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.731886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.747484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473a76f0-7e5c-461a-b89f-07c85c4a8d91 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.788619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.789382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.789778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.790239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.807192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:09.815233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:09.816119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:09.816894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:09.817491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 40/81] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:09.823658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.841058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc902d8-5b50-4376-a6f3-828ee221ee8f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.879011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.880018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.880477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.880925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.882818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:09.885684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:09.888450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 42/82] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:09.889716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:09.917727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.929714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:09.940742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a04daa-e024-43f7-b0c5-033df10081d1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:09.975764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:09.976388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:09.976665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:09.976937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:09.997404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f135b4-c339-4c23-bc4d-87c2b5756d10 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.041294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.042307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.042593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.043208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.075913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.078698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.163254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.165310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.212984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.216175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.249158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 41/83] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:10.250936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:10.328138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 43/84] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:10.329457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:10.436651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e742a5-a626-4306-abac-ba8e4bca419a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.462758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0cad9-13aa-4284-8f89-7a6cda1834c6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.481299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.482257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.482669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.483030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.483966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.485696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.506595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.507465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.507904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.508285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.509100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:10.509522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.511997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.515925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.523490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 42/85] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:10 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:10.523490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:10.533676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 44/86] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:10.533676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:10.577759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.667001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bad5a3-4cdf-4261-bfc3-af6af0e9086e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.685556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f55b6-0322-4bb3-9b25-00f4bb074962 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.711054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.711842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.712300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.712721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.731648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.732282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.732560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.732836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.734211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.735823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.737910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 45/87] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:10 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:10.743884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:10.824073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.825991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:10.861110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.888067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:10.930994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d9e80-b305-4706-ac98-015ddd838dfa None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:10.992645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:10.993578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:10.993981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:10.994380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:10.996420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:10.998890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.001657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 46/88] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:10 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:11.002807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.022706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 43/89] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:11.024782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.187459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537b716-51d4-48a9-bb56-9e60f028031d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.199607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e46c8ee-1069-46aa-b97c-9138baf30ae4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.246036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.246853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.247233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.247580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.249202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:11.252098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.255318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 47/90] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:11.257014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.264280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:11.268021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.268738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.269063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.269486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.270284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:11.270685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:11.272423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.281718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:11.283962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 44/91] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:11.287766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.482459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6292f1a-1f07-45d8-b148-e3402b7d2bb0 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.553557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a147fd-4ee9-4b8f-9f6c-9df90d23d5c4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.564783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.565454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.565735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.566008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.569100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:11.570626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.572775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 45/92] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:11.573351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.610694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.610903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.611879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.612169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.613397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:11.613397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:11.615104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.629438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 48/93] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 259 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:11.637046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:11.724623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:11.747594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:11.813299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe6c8b-96f3-4b28-a645-f9cfd59347f3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.834316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825aca9-570f-4dfc-bf46-4ec38abb0a06 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:11.934097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.935125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.935593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.936027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.941053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=69e577935e2a4950900c965d118c5e00) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:11.941053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:11.941654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:11.941654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:11.941654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:11.941654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:11.942213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:11.956850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 49/94] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/role_assignments?user.id=69e577935e2a4950900c965d118c5e00&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:12.101466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:12.104373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:12.281855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:12.358957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 46/95] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 786 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:12.370795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:12.613543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba8b9f-4a87-415e-ab44-5592a0314bef None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:12.677898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:12.684165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:12.685854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:12.686758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:12.687625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] PATH_INFO: `/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:12.690321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d3ba142a3e4d01ad4a47fc212bcee1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:12.693223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bffdc8-2de9-40b4-8afc-30100ddbd538 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:12.696863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 50/96] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:12 2022] GET /identity/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1 => generated 312 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:12.705721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:12.847836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9fcdb-c9d2-4c0b-9f6d-af52893dc62d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:12.890784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:12.891351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:12.891633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:12.891907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:12.892655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=a9d3ba142a3e4d01ad4a47fc212bcee1, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:12.892963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:12.894435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ef448-81fc-4894-8792-3db0d29b27d4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:12.900756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 47/97] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:12 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=a9d3ba142a3e4d01ad4a47fc212bcee1&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:13.739054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86bc3-7c5d-4b8a-9bec-6b7288167631 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:13.739598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86bc3-7c5d-4b8a-9bec-6b7288167631 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:13.739598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a86bc3-7c5d-4b8a-9bec-6b7288167631 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:13.740988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 51/98] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:13.747495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097358-608d-4e48-b039-5625edb01624 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:13.747994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097358-608d-4e48-b039-5625edb01624 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:13.748433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1097358-608d-4e48-b039-5625edb01624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:13.789987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-d1097358-608d-4e48-b039-5625edb01624 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:13.792201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d1097358-608d-4e48-b039-5625edb01624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:13.845240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:13.934619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 48/99] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:13.968611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a86bc3-7c5d-4b8a-9bec-6b7288167631 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:14.101675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a86bc3-7c5d-4b8a-9bec-6b7288167631 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:14.146032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:14.147067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.147587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.148065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.168027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-1b176927-2422-43ea-9bcc-297c85e87448 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:14.173213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:14.174052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:14.174798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:14.177383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 52/100] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:14.188921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097358-608d-4e48-b039-5625edb01624 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:14.343194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1097358-608d-4e48-b039-5625edb01624 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:14.408445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:14.409470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.409956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.410430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.411635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] RBAC: Adding query filter params (name=reader) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:14.412160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:14.414690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:14.422255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03963-46ef-43eb-829f-bb06d7805dc5 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.422532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03963-46ef-43eb-829f-bb06d7805dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.422813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc03963-46ef-43eb-829f-bb06d7805dc5 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.424909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 53/101] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:14.430076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521acb-712e-410e-85b6-9b5160efc571 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.430369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521acb-712e-410e-85b6-9b5160efc571 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.430655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05521acb-712e-410e-85b6-9b5160efc571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.436238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 49/102] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:14.442488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:14.464638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-05521acb-712e-410e-85b6-9b5160efc571 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:14.465891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-05521acb-712e-410e-85b6-9b5160efc571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:14.508654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:14.599367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:14.638101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 54/103] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:14.638416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b84155-c73a-42d0-9ff0-43430df2d6ea None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:14.639589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8c7e4-46a4-45c7-9c6c-a0f906e207d6 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.639880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8c7e4-46a4-45c7-9c6c-a0f906e207d6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.640160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae8c7e4-46a4-45c7-9c6c-a0f906e207d6 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.641811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 55/104] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:14.647453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.647739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.648019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.680555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:14.682054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:14.701627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:14.702711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:14.703239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:14.703716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] PATH_INFO: `/v3/users/b11348454f8c418d97f45ed916b31404` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:14.739309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] RBAC: Authorizing `identity:get_user(user_id=b11348454f8c418d97f45ed916b31404)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:14.748679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:14.753388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 50/105] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:14 2022] GET /identity/v3/users/b11348454f8c418d97f45ed916b31404 => generated 289 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:14.755050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:14.862201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 56/106] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 08 13:21:14.863244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:14.902273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:14.938007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ce4e6-024c-4643-9948-016b07fe9bfd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.009786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.011805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.012669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.013361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.015652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:15.019877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:15.023033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 51/107] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:15.024920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.132146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f24d3-7551-407e-80da-6c7a57f6dbaa None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.179776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.185895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d411cd4-f26b-4bae-8bc5-d274f85ee0e3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.226342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.227728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.228246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.228723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.242597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.243700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.244263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.244840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.246625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:15.256207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:15.257217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:15.258021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:15.265572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 52/108] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:15.266840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.299464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.360911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:15.362530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:15.424569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd62c5d-2f8a-4175-b572-559927228dc9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.471754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.472867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.473067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.473416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.474257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:15.476175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c14598-1e70-448a-ab1c-7d310dac2711 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:15.496003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 57/109] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:15.496003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.510777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 53/110] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:21:15 2022] POST /identity/v3/projects => generated 302 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:15.512126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9268a62-108c-42de-aec2-82a0ce398708 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.512413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9268a62-108c-42de-aec2-82a0ce398708 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.512689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9268a62-108c-42de-aec2-82a0ce398708 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.519197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 54/111] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:15.519197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec103-e0d4-404e-a88a-3723c2aab177 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.519197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec103-e0d4-404e-a88a-3723c2aab177 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.519197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ec103-e0d4-404e-a88a-3723c2aab177 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.519197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 55/112] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:15.522421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec103-e0d4-404e-a88a-3723c2aab177 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.531463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.623094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.643394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c3cc05-e15b-44c6-a3b4-0064477aa08c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.680862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ec103-e0d4-404e-a88a-3723c2aab177 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:15.683293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.683933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.684211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.684480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.685174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:15.685476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:15.686907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392918aa-ad09-430e-a2e8-1640ef5ecf1c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:15.698717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 58/113] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:15.704254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645fd-5335-43c5-b646-3a0c3900f038 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.704254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645fd-5335-43c5-b646-3a0c3900f038 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.704254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659645fd-5335-43c5-b646-3a0c3900f038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.720480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-659645fd-5335-43c5-b646-3a0c3900f038 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:15.722165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-659645fd-5335-43c5-b646-3a0c3900f038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:15.733323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:15.733990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.734266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.734928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.735844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:15.737334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a4538-10e2-4fa4-968b-1ce5e841c133 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:15.739280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 56/114] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:15 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:15.740307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.740610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.740888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.766269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:15.767677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:15.809809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 59/115] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:15.820602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15311-0419-4054-8923-b34df23d89c7 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:15.820898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15311-0419-4054-8923-b34df23d89c7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:15.821180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f15311-0419-4054-8923-b34df23d89c7 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:15.822653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 60/116] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:15.823492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f15311-0419-4054-8923-b34df23d89c7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.847010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.887692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 57/117] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:15.889031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:15.960865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:15.998240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f15311-0419-4054-8923-b34df23d89c7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.016328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ecc10b-02e0-421e-8752-4e0805b073ed None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.042118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.042767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.043053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.043352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.044592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.046103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3715fb-7fc5-4286-9f33-0cf01e927428 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.053506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 61/118] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:15 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.053506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.053506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.053506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.064233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.064848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.065141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.065420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.066178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=b11348454f8c418d97f45ed916b31404) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:16.066482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.068044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.073300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:16.074526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:16.076731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 58/119] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:15 2022] GET /identity/v3/role_assignments?user.id=b11348454f8c418d97f45ed916b31404&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.077759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.189371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 62/120] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:16.190547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.203761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.243898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9277f5d7-6959-4e51-9309-651ed9be57ce None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.285203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.291007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.291735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.292022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.292302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.310844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:16.318903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:16.319853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:16.320631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:16.321213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 59/121] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:16.322005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.328096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ab9cf-e4ef-4fd9-bc41-a708b7772128 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.391842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.393045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.393486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.393931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.409244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:16.415912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:16.417067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:16.418040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:16.420867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 63/122] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:16.422263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.466991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060bfb14-a35b-4a36-a6df-ddcdea2ba359 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.512197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.513237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.514443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.515294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.516146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.517739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.519955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 60/123] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.521481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.540334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.555771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397b995-4cdf-4c9c-ba21-94c87abe9b27 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.598650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.602111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.603132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.603574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.603985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.615646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:16.620925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:16.621843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:16.622607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 64/124] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:16.623271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.662350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac97c05-62a9-4500-bdfb-bdb9fe0306fd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.704673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.705310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.705598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.705879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.706563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:16.706873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.708367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.719806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 61/125] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.720759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.741019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3b37d4-28d1-4d67-90e9-1d9f722cf659 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.771372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.784799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.786717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.787987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.788610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.790753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:16.791686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.793496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.805506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 65/126] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.806937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.856567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4f3f7-1c74-4f2f-b181-0c2bd105c14c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.871721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:16.895515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.896651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.897267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.897741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/users/0415b7f9c30e4b5880842ee378935c5c/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.900281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, user_id=0415b7f9c30e4b5880842ee378935c5c, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.902107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.917946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 62/127] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:16 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/users/0415b7f9c30e4b5880842ee378935c5c/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:16.919509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:16.922199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716b3a02-b942-4c1b-a44a-93f00d66739f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:16.971945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:16.972595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:16.972881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:16.973161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:16.973854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:16.974164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:16.975997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:16.985220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 66/128] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:16.986094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.040002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fcab8d-af8e-4fbd-8937-3a3e14e60dab None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.081672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.082785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.083300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.083766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.085321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.087270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.089444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 63/129] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.090587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.093830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.145107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.195779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9843deb-8eb9-4730-90a8-8374529e1f23 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.208219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e961fff-a8eb-426c-85b3-9463b5c6a45a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.244029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.245058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.245559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.246041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.247579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.249570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.251815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 64/130] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:17 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.252502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.253047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.253171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.253448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.253729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.255874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.257368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.259182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 67/131] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:16 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.268485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.364063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b40590-c817-4880-9478-96eadb63c0b3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.376011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.402674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.403742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.404211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.404715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.406053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.407755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.409929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 65/132] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:17 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.410895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.449548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e19233b-da17-4f6b-afe6-6a1e938c2e3c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.478501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.509921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.510960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.511270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.511544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] PATH_INFO: `/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.514184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d3ba142a3e4d01ad4a47fc212bcee1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.515763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.526251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 68/133] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:17 2022] GET /identity/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1 => generated 312 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.527227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.618188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ad3f9-6357-4217-a7a4-61637b6da454 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.654071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.655017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.655517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.655955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.657379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.658992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.661061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 66/134] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:17 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:17.662068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.712926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.756988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b5467-1389-4235-a524-2107cba53ce0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.769256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398256b9-6ab8-442f-b49b-3a084436ed24 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.805172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.806305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.806764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.807410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] PATH_INFO: `/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1/users/0415b7f9c30e4b5880842ee378935c5c/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.809776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9d3ba142a3e4d01ad4a47fc212bcee1, user_id=0415b7f9c30e4b5880842ee378935c5c, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.811449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.827992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 67/135] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:17 2022] PUT /identity/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1/users/0415b7f9c30e4b5880842ee378935c5c/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:17.829186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:17.834534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.835383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.835739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.836083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/users/0415b7f9c30e4b5880842ee378935c5c/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.838679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, user_id=0415b7f9c30e4b5880842ee378935c5c, role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.840352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132af190-9311-46b4-af5c-966c3838f53d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:17.886890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 69/136] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:17 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/users/0415b7f9c30e4b5880842ee378935c5c/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:17.951904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c18f50-7a8b-4448-b49e-d07f59919fd4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:17.961052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:17.991809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:17.992635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:17.992975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:17.993456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/users/69e577935e2a4950900c965d118c5e00/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:17.995788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, user_id=69e577935e2a4950900c965d118c5e00, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:17.997527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:18.013364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fb4e-a58d-4409-9a33-546b4947bdb7 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.013748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fb4e-a58d-4409-9a33-546b4947bdb7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.014138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814fb4e-a58d-4409-9a33-546b4947bdb7 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.015849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 70/137] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:18.018727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 68/138] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:17 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/users/69e577935e2a4950900c965d118c5e00/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:18.035040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.035443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.035788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.073196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:18.074473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:18.135747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:18.168623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 71/139] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:18.212509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:18.332805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95cda29-fcd7-42b4-9731-6be408a7588d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:18.375831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:18.376887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.377357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.377815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.379853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:18.381368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc188b1-c496-462e-bacd-3be800ed34d8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:18.383340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 69/140] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:18.392370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:18.392706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632792b-a4cd-49c1-8483-6d831e9cd0a6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.393900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632792b-a4cd-49c1-8483-6d831e9cd0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.394184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f632792b-a4cd-49c1-8483-6d831e9cd0a6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.395656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 70/141] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:18.402999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.403330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.403521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.440330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:18.441523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:18.489989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:18.576526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 71/142] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:18.591763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6b767-3f69-4730-a65a-c273c7fef6da None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:18.656870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:18.670290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:18.671334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.671823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.672301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.673242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:18.674945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:18.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 72/143] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:18 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:18.810932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d123a-61bc-4197-8830-c81e129da1a9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:18.866592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:18.867288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:18.867574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:18.867847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:18.884065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:18.889295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:18.890312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:18.891274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:18.893974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 72/144] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:18.898213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:18.980709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:19.076498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a22a6b-168d-46ce-a844-2e7cfc6b6d52 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:19.127916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:19.128587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.128875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.129156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.129865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:19.130265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:19.131894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:19.142116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 73/145] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:19.148366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:19.186962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:19.280254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8321e4-8e46-4055-9b16-3e6d085cf6b5 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:19.316385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:19.317045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.317327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.317597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] PATH_INFO: `/v3/users/b11348454f8c418d97f45ed916b31404` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.318799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] RBAC: Authorizing `identity:get_user(user_id=b11348454f8c418d97f45ed916b31404)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:19.320331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:19.322214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 73/146] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:19 2022] GET /identity/v3/users/b11348454f8c418d97f45ed916b31404 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:19.328062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:19.474212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:19.481157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8c21a-527e-4456-80a1-9958afbf049b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:19.534371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:19.534960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.535270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.535546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.539950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:19.541521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9793c-7392-45c0-bb28-2f1ef4af10fc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:19.543347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 74/147] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:19 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:19.549351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:19.650723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:19.661849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb0d83-2fb1-4e04-b536-919a1b43151f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:19.696356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d103f-4bc2-4ee1-bac4-d1f52969788f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.697434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d103f-4bc2-4ee1-bac4-d1f52969788f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.697434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d103f-4bc2-4ee1-bac4-d1f52969788f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.698283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 75/148] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:19.704288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.704588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.704842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.710152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:19.710856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:19.711159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:19.711432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/users/b11348454f8c418d97f45ed916b31404/roles/3d18bd6319274c7c8f05348d36f5fd6c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:19.716174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, user_id=b11348454f8c418d97f45ed916b31404, role_id=3d18bd6319274c7c8f05348d36f5fd6c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:19.717049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:19.729015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:19.730388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:19.741258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 74/149] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:19 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/users/b11348454f8c418d97f45ed916b31404/roles/3d18bd6319274c7c8f05348d36f5fd6c => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:19.824041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 76/150] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:19.850564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:19.964658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c334a53d-d60f-418c-b38c-4594b2121e3f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:20.001884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:20.002547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:20.002826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:20.003119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:20.016221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:20.021112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:20.021987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:20.023270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 75/151] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:20.028723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:20.148365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:20.155638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f61b2b8-9f06-4401-b026-3a3899d617b5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:20.205462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:20.206118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:20.206401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:20.206674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:20.207710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:20.208022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:20.210367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:20.224109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 77/152] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:20.230796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:20.255764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:20.417463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6a8ab-97df-44f2-8f97-a3a77b77f53f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:20.483015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:20.484115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:20.484592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:20.485035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:20.487425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:20.490223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:20.493643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 76/153] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:20 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:20.502684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:20.714408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157078ab-3557-4b14-bc34-d39443912f1e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:20.792626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:20.793657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:20.794140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:20.794621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:20.839184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:20.900425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:20.901956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:21.018538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 78/154] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:21.028061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:21.228597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:21.272822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49decd02-1067-4d90-98bb-416a8279fedd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:21.339040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:21.340476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.341155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.341837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.344202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:21.347056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c95fbb-9127-4864-9804-ed973316dc97 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:21.350463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 77/155] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:21 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:21.359928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:21.378116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791384f-a3da-4a0a-b848-0ac227111610 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.378951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791384f-a3da-4a0a-b848-0ac227111610 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.379661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5791384f-a3da-4a0a-b848-0ac227111610 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.382640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 78/156] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:21.410404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.411288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.412016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.461365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:21.463878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:21.479626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:21.591228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e55ea93-0e39-4532-9562-682d7b8aca7e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:21.613620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 79/157] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:21.660467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:21.661134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.661506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.661736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.662386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:21.662710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:21.664210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5192f-1bd8-4b78-9f7a-011bc43dc291 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:21.675214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 79/158] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:21 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:21.682119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:21.684447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42458-7c26-4f3e-80a6-266d0044178f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.684752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42458-7c26-4f3e-80a6-266d0044178f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.685043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e42458-7c26-4f3e-80a6-266d0044178f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.686682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 80/159] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:21.705066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:21.705452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:21.705643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:21.743777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:21.745012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:21.874419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 81/160] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:21.899934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:21.904556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:21.951876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb62ec-6e96-48be-a842-45c109c6a8dd None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.000216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.048411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.049136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.049421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.049717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.052423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3fb58-36c0-4f99-885c-1f95d42f5781 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.080187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:22.087652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.088308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.088604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.089320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:22.090843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:22.092541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:22.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 80/161] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:22.100927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:22.104175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:22.106654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:22.110232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 82/162] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:22.115280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.233921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d14193-37d7-42d3-81f4-eda7f5a68bd5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.246936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.276281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.277136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.277508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.277875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.278667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:22.279115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:22.280763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:22.289893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 83/163] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:22.296033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.411298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ec635-e95f-4efe-969d-1d9ad352e5d9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.433157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0824f-d2ae-405c-9a73-a26a8d7e0cb5 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.452680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.453506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.453892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.454256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.456924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:22.458608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:22.460438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 84/164] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:22.466384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.521137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.533317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.533948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.534235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.534492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.539314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:22.539674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:22.541111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:22.556115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.562583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 81/165] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:22.570435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.611270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ff0cf-a63f-42a4-b39a-856159adbf53 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.650386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.651014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.651315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.651581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.767539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e097cb1-9cf5-4943-ba04-1a3dbbf69d60 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.785847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:22.788859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08cbbc-edd9-4267-8a8e-68921b8a72c0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:22.820573 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.821264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.821576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.821845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] PATH_INFO: `/v3/users/69e577935e2a4950900c965d118c5e00` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.823020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] RBAC: Authorizing `identity:get_user(user_id=69e577935e2a4950900c965d118c5e00)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:22.824542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af48619a-8527-4cf8-93ae-cfb44cef672e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:22.826491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 82/166] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/users/69e577935e2a4950900c965d118c5e00 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:22.827662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be8cd-300f-46c9-8d67-7fc4f0b4144c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.827936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be8cd-300f-46c9-8d67-7fc4f0b4144c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.828209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5be8cd-300f-46c9-8d67-7fc4f0b4144c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.829668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 83/167] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:22.833245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be8cd-300f-46c9-8d67-7fc4f0b4144c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:22.873920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.911161 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:22.950326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5be8cd-300f-46c9-8d67-7fc4f0b4144c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:22.987305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 85/168] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:21:22.989103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:22.989740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.989852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:22.990071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.990247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:22.990338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:22.990799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:23.023936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:23.025885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:23.073661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:23.075106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:23.150348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:23.154625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 86/169] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:23.159145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.205680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 84/170] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:21:23.212081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.302281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:23.328917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60699a1-e96d-4cce-bd1a-34f0d1aa0745 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:23.368144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:23.369156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:23.369904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:23.370398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:23.371853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:23.373541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:23.375614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 85/171] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:23.376663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.437376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:23.466486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26ecdc-052d-4a76-b719-356b0044c8cc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:23.581628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd238a8-169e-4855-bc92-ebc51c2cd4ea None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:23.621409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:23.622216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:23.622621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:23.622987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:23.635969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:23.641260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:23.642380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:23.643495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:23.645312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 86/172] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:23.645434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:23.646135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.646535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:23.646878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:23.647771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:23.649615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:23.652700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:23.664527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 87/173] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 509 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:23.666008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.772601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4362cb3-80df-4c96-bfc4-ee4b3814bfd2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:23.791498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:23.812564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:23.813405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:23.813782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:23.814170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:23.815037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=69e577935e2a4950900c965d118c5e00) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:23.815488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:23.817102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:23.824692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 87/174] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/role_assignments?user.id=69e577935e2a4950900c965d118c5e00&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:23.832406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:23.986897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aeffd5-a121-441e-9890-d30f02a572f0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:24.001789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a8160-6641-46ba-ad96-62b04d8e62cb None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:24.059044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:24.059709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.059986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.060252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.061011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:24.061315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:24.062828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935036fe-49cd-4d44-b1d3-88d1d09a16a6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:24.063853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:24.064500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.064784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.065062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.065756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:24.066063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:24.067515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:24.069887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 88/175] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:24.071042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73bdd9-1cb2-4375-9692-fc67acf8431c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.079733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73bdd9-1cb2-4375-9692-fc67acf8431c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.080073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73bdd9-1cb2-4375-9692-fc67acf8431c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.082005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 89/176] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:24 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:24.084068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 88/177] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:24.084472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:24.089434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.089716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.089996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.093906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:24.154368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:24.155705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:24.219521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:24.326219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 90/178] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:24.347274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0689d-3b1e-4938-bc65-648f667009a6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:24.354262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:24.395994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:24.396634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.396916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.397193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] PATH_INFO: `/v3/users/0415b7f9c30e4b5880842ee378935c5c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.398399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] RBAC: Authorizing `identity:get_user(user_id=0415b7f9c30e4b5880842ee378935c5c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:24.402781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:24.405030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 89/179] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:24 2022] GET /identity/v3/users/0415b7f9c30e4b5880842ee378935c5c => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:24.414809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:24.511521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:24.558282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b42040-0d81-4d60-9e86-4bc0f2c9046d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:24.567836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75589408-3c03-4804-a06a-5275ea10a150 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:24.611469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:24.612061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.612345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.612760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.629389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-70cc4a56-c4bd-4789-a83a-47176986013f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:24.635584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:24.636484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:24.637262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 91/180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:24.638451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f8b0-9f07-4991-953e-d30e8650a8b6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.638451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f8b0-9f07-4991-953e-d30e8650a8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.638746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27f8b0-9f07-4991-953e-d30e8650a8b6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.639894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 92/181] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:24.648159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.648459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.648605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:24.648805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.649853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.650414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.650908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.671136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:24.672389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:24.686981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:24.750030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:24.751908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649481c-9877-461b-972d-94d08135fa15 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:24.790763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 93/182] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:24.797989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:24.842819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:24.888910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 90/183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:24.889993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6233a5-845b-4681-a9b2-4c0a54ef213e None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:24.890267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6233a5-845b-4681-a9b2-4c0a54ef213e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:24.890556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6233a5-845b-4681-a9b2-4c0a54ef213e None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:24.896228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 91/184] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:24.896228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6233a5-845b-4681-a9b2-4c0a54ef213e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:24.954326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d89c187-9416-4347-a5e8-82afa146cee7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.001918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.002978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.003388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.004513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.005820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:25.006872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.010594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb42df-4806-4660-8d5c-c9080ce22929 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.033503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 94/185] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.033503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c9ee-f125-4af5-bb6d-4807ae5298b4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.033503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c9ee-f125-4af5-bb6d-4807ae5298b4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.033503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b85c9ee-f125-4af5-bb6d-4807ae5298b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.035517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6233a5-845b-4681-a9b2-4c0a54ef213e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.051767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.059998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4b85c9ee-f125-4af5-bb6d-4807ae5298b4 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:25.061491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4b85c9ee-f125-4af5-bb6d-4807ae5298b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:25.072206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.072850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.073134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.073911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.086086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:25.090836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:25.092118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 92/186] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:25.093828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.125695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.144437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 95/187] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:25.145669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19010f7-cea2-4ebf-8e33-6ff43e69c388 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.145943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19010f7-cea2-4ebf-8e33-6ff43e69c388 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.146217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19010f7-cea2-4ebf-8e33-6ff43e69c388 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.147706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 96/188] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:25.148222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19010f7-cea2-4ebf-8e33-6ff43e69c388 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.223243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2773e7-e941-4231-887a-77005f4a55ba None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.263655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19010f7-cea2-4ebf-8e33-6ff43e69c388 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.268723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.269400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.269680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.269951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] PATH_INFO: `/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.271328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] RBAC: Authorizing `identity:get_project(project_id=a9d3ba142a3e4d01ad4a47fc212bcee1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.272885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.274915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 93/189] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/projects/a9d3ba142a3e4d01ad4a47fc212bcee1 => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.274915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.315668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.316755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.317232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.317666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.322964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.339282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.340822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a3d29-d46a-4c2e-bc25-ea9ce7499250 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.342632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 97/190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.349583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.349987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.350367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.372325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:25.373574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:25.394333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.408147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1dccd6-d3bc-4bfb-8adf-0d79927c75b3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.443540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.444171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.444453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.444730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.445415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:25.445719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.447261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.452102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 98/191] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:25.452773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.456734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 94/192] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.460094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.585971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d19983-d176-44e3-9dd4-1d57a158db2d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.598558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b12174-c5ee-4b58-aa91-20d1fb47b69e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.613241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.625973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.626624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.626900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.627210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.640035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:25.648373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:25.649366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:25.650178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:25.656382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 99/193] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:25.656382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.658942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.659655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.660020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.660436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.661293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=a9d3ba142a3e4d01ad4a47fc212bcee1, user.id=0415b7f9c30e4b5880842ee378935c5c) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:25.661731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.663381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.671072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 95/194] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/role_assignments?user.id=0415b7f9c30e4b5880842ee378935c5c&scope.project.id=a9d3ba142a3e4d01ad4a47fc212bcee1&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.672513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.717342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:25.823907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715f0fa-2a36-4173-a93a-fff88198a8e4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.838087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657213bd-ea34-4851-9f07-8c3063a6f073 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:25.889728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.890583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.891038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.891486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.903820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:25.904830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:25.905262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:25.906454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:25.908547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:25.911952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:25.914619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 96/195] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:25.917279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:25.996392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.018739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.021632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.052552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.068768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d421b11-a750-4a0e-b3d5-1fc6c8b654c4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.107700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.108356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.108635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.108897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.110088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.111644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.113283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 97/196] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:26.114090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.181347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 100/197] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:26.182558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.243341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a88d11-2493-49fc-930b-dc0ba0e80fad None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.289115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.289894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.290201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.290477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.291312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:26.291753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.293497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.303233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.303820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 98/198] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:26.304702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.331352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.331567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576fab-00cd-4091-972c-255243d0f8a4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.369447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.370165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.370696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.371174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.455303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.456776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.458815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560bb07-b653-4472-8554-0a098ee272d1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.510854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.511954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.512449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.512922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.514071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.516742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.561087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 99/199] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:26 2022] POST /identity/v3/users => generated 297 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:26.561087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.562432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.599309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 101/200] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:26.600881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.712835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.753183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e65702-7db0-4fa9-9f6f-e1000f8ade17 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.783425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065d71d-c9d1-4402-a231-7d1363c66b00 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:26.815726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.816778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.817256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.817683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.819980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.823198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.826406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 100/201] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:26.827935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.836877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:26.838407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:26.839174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:26.839935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] PATH_INFO: `/v3/users/b11348454f8c418d97f45ed916b31404` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:26.842366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] RBAC: Authorizing `identity:get_user(user_id=b11348454f8c418d97f45ed916b31404)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:26.846767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:26.850409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 102/202] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/users/b11348454f8c418d97f45ed916b31404 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:26.852180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:26.954688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:26.998802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778bdab-7576-4b4a-9420-0445410473e7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:27.029022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630c43db-146d-46a0-93c7-dd3a59fe9463 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:27.054118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:27.054783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:27.055064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:27.055370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:27.057010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:27.058163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:27.060597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 101/203] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:27.060871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:27.089964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:27.101633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:27.102777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:27.102777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:27.102777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:27.103681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=310b41c6d824433db1d45b6b87e64bd0) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:27.103989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:27.105553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:27.112402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 103/204] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:26 2022] GET /identity/v3/role_assignments?user.id=310b41c6d824433db1d45b6b87e64bd0&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:27.120228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:27.280554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad85e9a-d6c6-4c83-a056-6d6ea2d13e97 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:27.301406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:27.324504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d998c8-39b7-4c8c-9984-d54bbdf94592 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:27.324753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:27.325284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:27.325565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:27.325837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:27.326600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=310b41c6d824433db1d45b6b87e64bd0) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:27.326911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:27.328480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:27.335035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 104/205] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:27 2022] GET /identity/v3/role_assignments?user.id=310b41c6d824433db1d45b6b87e64bd0&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:27.368419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:27.369364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:27.369799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:27.370225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:27.465286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:27.466898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:27.494386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:27.658241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 102/206] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 597 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:27.664276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:27.790371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69fc246-a792-4d40-89fe-379625407aab None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:27.826765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:27.827624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:27.831121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:27.831121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:27.831121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:27.831121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb709d6-2cad-4587-86c8-b6f378355cbb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:27.832827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 105/207] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:27 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:27.845033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:28.010056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b0112-9e58-4f52-8f2b-796c5a6aea71 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:28.030284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:28.058586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:28.059262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:28.059543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:28.059819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:28.060578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, scope.project.id=6249ead7798a4166a3569d19a133c509, user.id=b11348454f8c418d97f45ed916b31404) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:28.060874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:28.062356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218e2d23-bb60-466b-aa21-e15fbf80e569 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:28.074867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 103/208] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:27 2022] GET /identity/v3/role_assignments?user.id=b11348454f8c418d97f45ed916b31404&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:29.852337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce2bf-3a11-4df9-8c10-19dbc9a511f2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:29.853359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce2bf-3a11-4df9-8c10-19dbc9a511f2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:29.853533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce2bf-3a11-4df9-8c10-19dbc9a511f2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:29.855312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 106/209] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:29.856535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd7a8a-cc4e-404f-851a-7bb5dfceaddd None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:29.857250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd7a8a-cc4e-404f-851a-7bb5dfceaddd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:29.857250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd7a8a-cc4e-404f-851a-7bb5dfceaddd None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:29.858747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 104/210] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:29.864335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:29.864712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:29.864888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:29.871040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:29.871330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:29.871637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:29.889386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:29.890615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:29.902974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:29.905950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:29.939989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:29.965856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 107/211] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:30.007582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.012073 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 105/212] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:30.035948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.172521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c583a-6c88-4b57-8d7d-d7c9bc9530d7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.180696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:30.194610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc6d219-b097-4c3e-97c9-249b6fcc63c3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.211543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.212243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.212519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.212791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.224571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-5f15c9cb-ca07-41ae-9f1d-630eaa69a484 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:30.229525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:30.230450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:30.231325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:30.235770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 106/213] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:30.237013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac258-237f-439d-9d3c-9e630b4165ad None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.237301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac258-237f-439d-9d3c-9e630b4165ad None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.237571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624ac258-237f-439d-9d3c-9e630b4165ad None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.243203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 107/214] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:30.243203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ac258-237f-439d-9d3c-9e630b4165ad None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.265400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.266682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.267200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.267847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.288682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1ef9032e-6ddc-4d8f-bc79-0383c7bd115b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:30.299184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:30.300153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:30.300975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:30.301574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 108/215] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:30.304100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.304100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.305346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.329441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:30.331413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:30.362324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624ac258-237f-439d-9d3c-9e630b4165ad None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.371584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:30.423194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 109/216] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:30.423194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.432667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.433818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.434340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.435199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.436297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:30.436897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:30.439392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:30.454823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 108/217] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:30.456586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.501330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:30.592062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f964d07-d334-4702-a534-3350bb39b9f9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.617939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597c-8910-4987-bcab-6c938244c88d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.635728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.636368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.636647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.637018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.637785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:30.637951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:30.639388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:30.654569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 110/218] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:30.654569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.673181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.674367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.674754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.675170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.677025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:30.679540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:30.682125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 109/219] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:30.683372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.712142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:30.816664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb43aff-f9ac-468f-817b-11d0e48bbdf2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.824451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:30.832203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1f909-282f-41b1-adc8-fe73781cee8c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:30.871103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.872134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.872418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.872688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] PATH_INFO: `/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.880845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:30.881863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.882270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.882672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.884463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:30.886619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:30.889198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 110/220] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:30.890078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:30.894393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] RBAC: Authorizing `identity:get_user(user_id=2e6a575b0f354e60ab94c97eb3a6c99c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:30.896554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c289ab1-906b-44f5-b0db-1f6b4df706fd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:30.899500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 111/221] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c => generated 297 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:30.901620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85001800-e41c-49df-b393-854d0ba18898 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:30.902091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85001800-e41c-49df-b393-854d0ba18898 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:30.902569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85001800-e41c-49df-b393-854d0ba18898 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:30.905048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 112/222] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:30.906411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85001800-e41c-49df-b393-854d0ba18898 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.029136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a53559-e1dd-4b93-9716-e11726122963 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:31.031798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:31.053113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85001800-e41c-49df-b393-854d0ba18898 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:31.073867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:31.074573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.074915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.075316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.076155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:31.078108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:31.103476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:31.104472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.104923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.105357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.107286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:31.107875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 111/223] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:30 2022] POST /identity/v3/users => generated 297 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:31.109050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.109848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa22606-8b8b-4977-8df6-3a57cffd3464 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:31.112802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 113/224] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:30 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:31.114825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.115342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.115676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.151834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:31.156430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:31.158716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:31.254031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 114/225] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:31.255491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.308099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fb9a8-b1e6-44f5-a77f-d20367c8a485 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:31.373167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:31.374209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.374668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.375174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.422033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:31.450743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331231a1-b384-4853-a44a-472f6369ac04 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:31.504871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:31.505575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.505843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.506107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/310b41c6d824433db1d45b6b87e64bd0/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.508537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, user_id=310b41c6d824433db1d45b6b87e64bd0, role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:31.510470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:31.515495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:31.517024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:31.532468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 115/226] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:31 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/310b41c6d824433db1d45b6b87e64bd0/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:31.533124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.572131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:31.733480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618798d-ac6d-4a68-81e4-ba2e42a8585b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:31.763183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 112/227] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:31.786488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.817183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:31.817859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:31.818138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:31.818409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:31.835774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:31.842901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:31.843919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:31.844705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 116/228] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:31.854206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:31.868208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:31.995111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59118acf-b39a-4a6d-9818-03626e917b94 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:32.018317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:32.036918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:32.038059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:32.038543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:32.039027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:32.045143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:32.047125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:32.050141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 113/229] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:31 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:32.057085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c2c46-21bd-4acb-9192-ffa123f5b165 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:32.058859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:32.139152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:32.139645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:32.139921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:32.140189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:32.140889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:32.141195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:32.142634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:32.169862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 117/230] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:32.178004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:32.202199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d68406-ff7c-4a6c-ae7f-7b269bcb0ef3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:32.251844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:32.252974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:32.253549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:32.254011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:32.256014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=2e6a575b0f354e60ab94c97eb3a6c99c) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:32.256644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:32.259638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:32.273448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 114/231] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:32 2022] GET /identity/v3/role_assignments?user.id=2e6a575b0f354e60ab94c97eb3a6c99c&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:32.395210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:32.477575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184135fb-82e5-4538-8f5d-7149265fb2fc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:32.533640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:32.534698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:32.535233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:32.535699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:32.538581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:32.540680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:32.542675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 118/232] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:32 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:32.548688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:32.671168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:32.789531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba1617-9ba0-4952-b0a1-c96fbf57e107 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:32.870556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:32.871671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:32.872153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:32.872626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:32.874608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:32.877322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dd8e39-ba87-4b9a-990f-80821f140a5a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:32.880756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 115/233] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:32 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:32.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:33.127479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1907b408-45d3-4f41-88d5-6544e20807d4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:33.208836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:33.210033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:33.210526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:33.211022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/310b41c6d824433db1d45b6b87e64bd0/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:33.217821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, user_id=310b41c6d824433db1d45b6b87e64bd0, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:33.220995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a276ce2d-4315-4ec5-bcdf-9c433ae835a4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:33.233350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:33.261821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 119/234] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:32 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/310b41c6d824433db1d45b6b87e64bd0/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:34.572865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bf5d-e9a2-4026-a6db-6f8c0eca56fd None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:34.573655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bf5d-e9a2-4026-a6db-6f8c0eca56fd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:34.573655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241bf5d-e9a2-4026-a6db-6f8c0eca56fd None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:34.574810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 116/235] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:34.580800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:34.581578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:34.581578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:34.608974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8cf3-b3de-4c2e-a777-bf457b5c522f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:34.609272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8cf3-b3de-4c2e-a777-bf457b5c522f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:34.609563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8cf3-b3de-4c2e-a777-bf457b5c522f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:34.610897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 117/236] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:34.612420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:34.613669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:34.616487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:34.616802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:34.617052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:34.651222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:34.652474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:34.746345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 120/237] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:34.788469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 118/238] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:34.797783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:34.811665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:34.937128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:34.954080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93dca3-76c7-4b01-afa1-50e883191015 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:34.985567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:34.997448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:34.998356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:34.998778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:34.999226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.012564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-30f1a4db-8c99-4039-9479-d1de0100c406 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:35.017921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:35.018957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:35.019872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:35.021856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 121/239] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:35.027479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792b55-0186-4064-a6b5-dd89a7476c4f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.028068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792b55-0186-4064-a6b5-dd89a7476c4f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.028484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792b55-0186-4064-a6b5-dd89a7476c4f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.030121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 122/240] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:35.032689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75792b55-0186-4064-a6b5-dd89a7476c4f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.065713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09c2cdb-38b7-4828-94d8-273defb28f54 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.134421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.135776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.136204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.136620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.150751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-09132843-f7d4-4583-8cca-8ac17cd2bdf3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:35.156158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:35.157148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:35.157954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:35.160604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 119/241] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:35.162447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.162901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.163338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.186318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:35.188057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:35.256127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75792b55-0186-4064-a6b5-dd89a7476c4f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.269143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 120/242] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:35.270685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.317651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:35.332484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.333515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.333996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.334449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.335618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:35.336137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:35.338735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:35.354856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 123/243] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:35.359247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.395022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870e726-50d3-473c-9230-48a788d5cb5d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.433638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.434430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.434813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.436035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:35.436439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:35.438024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:35.448513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 121/244] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:35.449504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.459609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:35.557972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48090ac9-b062-4eba-9aa8-9bd4d48b4496 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.566142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afc30a-ea97-49c5-af86-c97d73b24895 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.568005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:35.607528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.608585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.608735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.609106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.611222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:35.614692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:35.623344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 122/245] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:35.624643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.629220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.634073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.636463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.637231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.643109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:35.646391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:35.653086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 124/246] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:35.654457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.768647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be705e5f-f314-466c-b833-7ffcd71e9651 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.831626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.832520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.832938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.833321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] PATH_INFO: `/v3/users/32810df2fa0c4913b0c49e5518e06e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.835916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60ea833-50a9-4587-ada4-a8bbec0eeb87 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:35.860611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] RBAC: Authorizing `identity:get_user(user_id=32810df2fa0c4913b0c49e5518e06e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:35.863815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:35.867137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 123/247] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/users/32810df2fa0c4913b0c49e5518e06e00 => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:35.868547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:35.869638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:35.896749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:35.897760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:35.898029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:35.898301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:35.924814 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.029868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:36.031373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928038c4-5ad7-476e-b14e-fe6f4a98a509 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:36.051870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b10b72d-bb0f-4805-b79e-481e92a51975 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:36.093682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:36.094437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.094742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.095019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.096139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:36.097718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a95608-089f-47d2-8386-4bb736e95db0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:36.124062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 124/248] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:35 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:36.125641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238121a6-273b-4a5b-902a-7fe9856ddd2a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.125952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238121a6-273b-4a5b-902a-7fe9856ddd2a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.126396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238121a6-273b-4a5b-902a-7fe9856ddd2a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.135848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 125/249] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:36.135848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238121a6-273b-4a5b-902a-7fe9856ddd2a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:36.237685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 125/250] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:36.240288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.240769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.241241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.249935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.273340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238121a6-273b-4a5b-902a-7fe9856ddd2a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:36.281111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:36.283311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:36.308878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.319753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:36.320867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.321444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.321983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.407214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:36.407214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:36.426918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 126/251] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:36.428654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:36.518093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.541133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 126/252] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:36.542754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:36.636042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71857d92-1c9b-4bfa-a821-304c3fbd9e7a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:36.703350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:36.704020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.704305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.704584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.705833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:36.707570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:36.709629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 127/253] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:36.710135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:36.717505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6668f9-8bce-44f0-b713-8d0f63aa47e3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:36.736119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.785756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:36.786824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.787316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.787788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.808202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-924606e9-43b8-4835-b3be-7051d5689e41 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:36.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:36.819622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:36.821247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 127/254] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:36.822543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211c0b-3b1d-411b-8302-c6e98f5ac470 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.822543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211c0b-3b1d-411b-8302-c6e98f5ac470 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.823497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6211c0b-3b1d-411b-8302-c6e98f5ac470 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.831124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 128/255] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:36.831124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6211c0b-3b1d-411b-8302-c6e98f5ac470 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:36.847140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6596021-aeb7-4f40-a5f1-62acafb45ae6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:36.919056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:36.919975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:36.922512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:36.923365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:36.924122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:36.926589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:36.929913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:36.934769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 128/256] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:36.937061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:37.025483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6211c0b-3b1d-411b-8302-c6e98f5ac470 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:37.089547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:37.090475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.090960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.091469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.092926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=310b41c6d824433db1d45b6b87e64bd0) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:37.093520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:37.096512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac693401-8259-422d-8f37-586aa88b9aa6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:37.108455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 129/257] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/role_assignments?user.id=310b41c6d824433db1d45b6b87e64bd0&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:37.110027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:37.110822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.111888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.111888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.149535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96c32b-7ad9-4412-8969-e3d516263eb0 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:37.163313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:37.165362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:37.220662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:37.222809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.223602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.224321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.225738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:37.226543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:37.230367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:37.249365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 129/258] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:37.251745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:37.428613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 130/259] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:37.429554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:37.513505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80a1802-c17c-46dd-a86f-b33275d2ee65 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:37.527797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:37.593669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:37.594624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.595059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.595534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.596949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=32810df2fa0c4913b0c49e5518e06e00) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:37.597572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:37.605204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:37.615816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f195c3-6aa4-485d-8d8d-c7b478c1ae42 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:37.616538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 130/260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:37 2022] GET /identity/v3/role_assignments?user.id=32810df2fa0c4913b0c49e5518e06e00&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:37.618195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:37.641795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:37.701245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:37.702334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.702833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.703368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] PATH_INFO: `/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.709709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] RBAC: Authorizing `identity:get_user(user_id=2e6a575b0f354e60ab94c97eb3a6c99c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:37.713146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:37.716816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 131/261] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:37 2022] GET /identity/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c => generated 297 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:37.726047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:37.930238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e77e21-cb79-4c72-9c84-ad160a237d78 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:37.970987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:37.971643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:37.971926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:37.972203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:37.985276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:37.990481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:37.992996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:37.994756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 131/262] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:38.001312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:38.004663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b30475-48bf-49b7-9729-7ef53de3b2e7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:38.013357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:38.072183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:38.072857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:38.073134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:38.073397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:38.079891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:38.081338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:38.082932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:38.085148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 132/263] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:37 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:38.091898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:38.120671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca079fef-9c1a-4894-ad08-07b613ed7f72 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:38.193126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:38.193764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:38.194023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:38.194309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:38.195025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:38.195360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:38.196857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:38.220558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 132/264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:38.225142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:38.320283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0dfb05-9665-4072-b666-b75149d61930 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:38.404783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd2fd7-5f4e-442c-94d3-ad9e502ebad1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:38.407395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:38.408095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:38.408374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:38.408651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/2e6a575b0f354e60ab94c97eb3a6c99c/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:38.412709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, user_id=2e6a575b0f354e60ab94c97eb3a6c99c, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:38.414229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:38.447404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 133/265] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:38 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/2e6a575b0f354e60ab94c97eb3a6c99c/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:38.465496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:38.469954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:38.471214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:38.471214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:38.472272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] PATH_INFO: `/v3/users/310b41c6d824433db1d45b6b87e64bd0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:38.479221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] RBAC: Authorizing `identity:get_user(user_id=310b41c6d824433db1d45b6b87e64bd0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:38.479221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:38.479221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 133/266] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:38 2022] GET /identity/v3/users/310b41c6d824433db1d45b6b87e64bd0 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:38.484456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:38.550791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:38.647109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dcc152-e4cc-4cce-a33b-b5a4506290da None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:38.691549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:38.692195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:38.692477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:38.692748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:38.795376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:38.796852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:38.846465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:38.959565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 134/267] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:38.965511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:39.210059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c249d-1238-468d-9982-bb95294d5788 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:39.280726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:39.282281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:39.283009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:39.283754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:39.286048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:39.288962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41c42f-ffce-4d4f-b545-5484f22a337b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:39.292308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 134/268] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:38 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:39.299398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:39.492762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd852de1-2e89-42db-8df7-ec87c296e47b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:39.531166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:39.532061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:39.532484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:39.532895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:39.533820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=310b41c6d824433db1d45b6b87e64bd0) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:39.534331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:39.536081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d783c2-b253-4d60-aeb1-d1d4b7e3dd0a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:39.538552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:39.547304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 135/269] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:39 2022] GET /identity/v3/role_assignments?user.id=310b41c6d824433db1d45b6b87e64bd0&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:39.873014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008a04-1c09-40d1-9998-5ac721157b84 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:39.873329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008a04-1c09-40d1-9998-5ac721157b84 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:39.873951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64008a04-1c09-40d1-9998-5ac721157b84 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:39.876907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 135/270] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:39.886988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:39.888422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:39.889081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:39.921813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:39.924261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:40.022411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc34f8-e347-4886-bf22-2a9ec254b318 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.023263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc34f8-e347-4886-bf22-2a9ec254b318 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.023398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cc34f8-e347-4886-bf22-2a9ec254b318 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.030670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 136/271] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:40.036231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.037102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.037819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.072095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:40.074832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:40.087265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 136/272] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:40.133400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.213596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:40.228947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 137/273] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:40.272667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.366956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d9138-cec8-4070-9778-41e120c68c33 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:40.418399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:40.429607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:40.430970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.431694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.432231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.444488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:40.449710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:40.450639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:40.451468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:40.458902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 137/274] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:40.460286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.489007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f733bae-f138-482c-85a1-63870d7b0794 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:40.541124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:40.542196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.542690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.543220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.545348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:40.547919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:40.558816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 138/275] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:40.561509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.574823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc993b-f38c-4a2a-873e-8d346453d33d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:40.613097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:40.614237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.614706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.615188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.616060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:40.616601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:40.618237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:40.627763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 138/276] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:40.633917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.659049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:40.688653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:40.704346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a334c4-dec7-4cdf-9840-ddba22abd114 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:40.743396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:40.744022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.744302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.744576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.745255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:40.746796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:40.764484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158858dc-e51d-42a7-9814-48bf38a78369 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:40.783590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 139/277] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:21:40 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:21:40.808526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:40.809624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.810152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.810635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] PATH_INFO: `/v3/users/b031076105584f259adc62ac84d3f48e` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.828676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] RBAC: Authorizing `identity:get_user(user_id=b031076105584f259adc62ac84d3f48e)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:40.830645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafe7867-4756-40c6-95c3-2a85d5bd6aad None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:40.833323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 139/278] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:40 2022] GET /identity/v3/users/b031076105584f259adc62ac84d3f48e => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:40.840010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:40.935140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c42993-af3e-47dd-be11-58f638cc98d2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.935888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c42993-af3e-47dd-be11-58f638cc98d2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.935888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c42993-af3e-47dd-be11-58f638cc98d2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.937019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 140/279] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:40.942832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ebc5b-409a-421d-bbf1-14a2e6987dbc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:40.943267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ebc5b-409a-421d-bbf1-14a2e6987dbc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:40.943652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67ebc5b-409a-421d-bbf1-14a2e6987dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:40.964444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d67ebc5b-409a-421d-bbf1-14a2e6987dbc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:40.965742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d67ebc5b-409a-421d-bbf1-14a2e6987dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:40.998759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05e227-97db-4488-b2c2-f2617e8861ec None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.030115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:41.048839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 141/280] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:41.051978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.052500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c980-c3d5-47fc-9a63-49f6ac05037a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.052639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.052802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c980-c3d5-47fc-9a63-49f6ac05037a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.052931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.053057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b885c980-c3d5-47fc-9a63-49f6ac05037a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.053183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.053870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, user.id=b031076105584f259adc62ac84d3f48e, scope.system=all) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:41.054154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:41.054929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 142/281] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:41.055826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:41.062297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.062297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.062297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.071129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 140/282] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 13:21:40 2022] GET /identity/v3/role_assignments?user.id=b031076105584f259adc62ac84d3f48e&scope.system=all&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 226 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:41.072159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.084952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:41.086210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:41.150293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:41.206721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 143/283] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:41.234612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a346a2d-097f-45c3-9031-0ef4025e2146 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.234935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.292047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.292731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.293005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.293268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.307168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:41.312557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:41.314773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:41.316654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 141/284] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:41.320933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.411951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a9aa4-b187-4a2e-8b76-c54e01cc628d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.427878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:41.468269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.471037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.472038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.477459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.508150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:41.515636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:41.517559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:41.519757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 144/285] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:41.530291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.570988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a763fee-8214-4569-911a-6dde36690311 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.595671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:41.634061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.634704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.635375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.635799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.636500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:41.636875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:41.638579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:41.657753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 142/286] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:41.664370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.741805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d71a9d-2794-4b2a-abe1-df83bdfc8f94 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.788159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.789191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.789695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.790226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.791048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:41.791749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:41.794562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:41.815231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 145/287] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:41.836598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:41.862046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e727d38b-a3cd-479b-814a-f15148282d1f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:41.872474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:41.933334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:41.934022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:41.934306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:41.934580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] PATH_INFO: `/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:41.936750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] RBAC: Authorizing `identity:get_user(user_id=2e6a575b0f354e60ab94c97eb3a6c99c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:41.938446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:41.940376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 143/288] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/users/2e6a575b0f354e60ab94c97eb3a6c99c => generated 297 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:41.946866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:42.006255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:42.024889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30332069-f972-4c4e-b31f-7451cbaf9c6b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:42.108118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:42.109146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:42.109609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:42.110057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] PATH_INFO: `/v3/users/32810df2fa0c4913b0c49e5518e06e00` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:42.117001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] RBAC: Authorizing `identity:get_user(user_id=32810df2fa0c4913b0c49e5518e06e00)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:42.119731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:42.122895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 146/289] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/users/32810df2fa0c4913b0c49e5518e06e00 => generated 297 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:42.129413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:42.249399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9e6d4-9152-4e8e-920a-9e7327ef3e7f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:42.355290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:42.356326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:42.356800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:42.357258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:42.378749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb21c5-f7b3-484c-aa1d-cae694e148fd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:42.428624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:42.429463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:42.429747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:42.430021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:42.434312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:42.436016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:42.437907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 147/290] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:42 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:42.463429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:42.471731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:42.511512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:42.540788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:42.542275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:42.668588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b513af2-62f9-4beb-b0b6-3f2aa27c76b9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:42.788565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:42.789723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:42.790244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:42.790735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/32810df2fa0c4913b0c49e5518e06e00/roles/3d18bd6319274c7c8f05348d36f5fd6c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:42.794699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, user_id=32810df2fa0c4913b0c49e5518e06e00, role_id=3d18bd6319274c7c8f05348d36f5fd6c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:42.801643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:42.839800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 148/291] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:21:42 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/users/32810df2fa0c4913b0c49e5518e06e00/roles/3d18bd6319274c7c8f05348d36f5fd6c => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:21:42.846971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 144/292] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 901 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:42.856984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:43.060691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:43.088885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd7862-2a72-43cf-a6a6-c3c963a98c36 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:43.145865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:43.146534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.146812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.147104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.152042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:43.153553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcc2b48-7f80-440e-8826-483355eb41bd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:43.155823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 149/293] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:42 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:43.162050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:43.262726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:43.294487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fb779-a400-495e-a500-cebaf0a11a7f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:43.336884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:43.337422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.337698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.337954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.338702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=2e6a575b0f354e60ab94c97eb3a6c99c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:43.339007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:43.340485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc468d-c68e-4655-a39b-83857edf28c8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:43.350761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 145/294] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:43 2022] GET /identity/v3/role_assignments?user.id=2e6a575b0f354e60ab94c97eb3a6c99c&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:43.506018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577c7-956b-47f7-be84-a59062a76623 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.506518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577c7-956b-47f7-be84-a59062a76623 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.507008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8577c7-956b-47f7-be84-a59062a76623 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.510770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 150/295] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:43.517394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7a39-fd69-4eb4-80a6-d55b823ad897 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.517896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7a39-fd69-4eb4-80a6-d55b823ad897 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.518391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173a7a39-fd69-4eb4-80a6-d55b823ad897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.558601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-173a7a39-fd69-4eb4-80a6-d55b823ad897 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:43.561041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-173a7a39-fd69-4eb4-80a6-d55b823ad897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:43.723042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 146/296] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:43.747462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8577c7-956b-47f7-be84-a59062a76623 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:43.831964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f76a17-fa09-48fd-8e45-d456984c9cc9 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.832388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f76a17-fa09-48fd-8e45-d456984c9cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.832805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f76a17-fa09-48fd-8e45-d456984c9cc9 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.835262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 147/297] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:43.843675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:43.844084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:43.844471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:43.878512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:43.879810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:43.966530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8577c7-956b-47f7-be84-a59062a76623 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.004291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.008293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 148/298] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:44.042482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.044064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.044771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.045458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.047517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.071353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:44.083685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:44.085664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:44.087595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:44.092474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 151/299] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:44.101645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.138042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.192025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88e5cd-9a10-4f2a-9402-d099f1ef42ac None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.247344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.248976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.249816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.250519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.271154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:44.281393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:44.283181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:44.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:44.289164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 149/300] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:44.299832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.330868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d07a98-0389-419d-823f-9e9cf2782163 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.390811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.391478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.391747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.392014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.392694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:44.393278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:44.394683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38adaa62-6538-466b-8ced-7f48f64ee617 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:44.404328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 152/301] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:44.407393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62def-29d8-4513-bbb2-4e95f866ea13 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.408256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62def-29d8-4513-bbb2-4e95f866ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.409244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed62def-29d8-4513-bbb2-4e95f866ea13 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.415683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 153/302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:44.415683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62def-29d8-4513-bbb2-4e95f866ea13 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.436479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.513362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c8dd3-823b-4ca5-99e2-d7338ec3fa1a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.518042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.558559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed62def-29d8-4513-bbb2-4e95f866ea13 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.574690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.576077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.577133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.577765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.579037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:44.579765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:44.582174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2711a0-43fd-4d71-9352-00b44663aa10 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:44.599003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 150/303] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:44.599003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.599003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.599003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.616696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.617531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.617966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.618357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] PATH_INFO: `/v3/users/61f45aee715343a5b40a3b3e011c1676` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.632272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:44.633840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:44.641835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=61f45aee715343a5b40a3b3e011c1676)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:44.645416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:44.657244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 154/304] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/users/61f45aee715343a5b40a3b3e011c1676 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:44.657244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.731934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 151/305] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:44.733605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.814010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.838440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a71a1-bc18-42da-9d09-9b3a665abaf2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.893561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e41073-a792-43cb-999d-c2dc5e6d0245 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:44.898493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.899204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.899564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.899835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] PATH_INFO: `/v3/users/b031076105584f259adc62ac84d3f48e` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.901221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] RBAC: Authorizing `identity:get_user(user_id=b031076105584f259adc62ac84d3f48e)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:44.902753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:44.904921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 155/306] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/users/b031076105584f259adc62ac84d3f48e => generated 293 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:44.905420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:44.909030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:44.948633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:44.949555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:44.949980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:44.951144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:44.951828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, user.id=61f45aee715343a5b40a3b3e011c1676, scope.system=all) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:44.952233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:44.954140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:44.971113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 152/307] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/role_assignments?user.id=61f45aee715343a5b40a3b3e011c1676&scope.system=all&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:44.971113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:45.054986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b276554c-49a0-4266-bac1-5953c3dc0568 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:45.098721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46468b49-99ca-4f0a-add7-cd0ff02a7ea7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:45.107493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:45.108305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:45.109424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:45.109784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:45.111159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:45.113889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:45.116225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 156/308] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:45.118253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:45.126474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:45.146364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:45.147228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:45.147585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:45.147927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] PATH_INFO: `/v3/system/users/b031076105584f259adc62ac84d3f48e/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:45.149740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b031076105584f259adc62ac84d3f48e, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:45.151418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4edf23-028b-44f9-8798-de0edbc7174a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:45.166298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 153/309] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Aug 8 13:21:44 2022] PUT /identity/v3/system/users/b031076105584f259adc62ac84d3f48e/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:21:45.232890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:45.274745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030475f-719c-4a2e-9b3b-ce4d99df9761 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:45.328251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:45.329344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:45.329823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:45.330294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] PATH_INFO: `/v3/groups` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:45.331624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:45.334328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97f617-7eab-4b00-b1e2-1109f5147c73 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:45.363927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 157/310] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Aug 8 13:21:45 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:21:46.008063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19743-33d2-4e13-91f0-6febaf218456 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:46.009412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19743-33d2-4e13-91f0-6febaf218456 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:46.010259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a19743-33d2-4e13-91f0-6febaf218456 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:46.012213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 154/311] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:46.018246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:46.018878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:46.019397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:46.040543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:46.042208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:46.122062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 158/312] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:46.147548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19743-33d2-4e13-91f0-6febaf218456 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:46.339587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a19743-33d2-4e13-91f0-6febaf218456 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:46.406717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:46.407813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:46.408311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:46.408793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:46.428950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:46.439006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:46.441024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:46.443156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 155/313] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:46.448231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:46.555576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:46.603364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdea441-ff2a-4034-a8d3-479ba663f27a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:46.641299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:46.642016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:46.642298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:46.642568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:46.643303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:46.643609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:46.645107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:46.655475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 159/314] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:46.662212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:46.711322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:46.821540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668404b-7f24-4637-bd5a-dca878b01c29 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:46.916058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:46.917445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:46.918093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:46.918735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] PATH_INFO: `/v3/users/32810df2fa0c4913b0c49e5518e06e00` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:46.920905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] RBAC: Authorizing `identity:get_user(user_id=32810df2fa0c4913b0c49e5518e06e00)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:46.923976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:46.927782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 156/315] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:46 2022] GET /identity/v3/users/32810df2fa0c4913b0c49e5518e06e00 => generated 297 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:46.933581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:47.046766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c6ec0c-1855-4f84-a6fb-a14fa22f846d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:47.075671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:47.085173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:47.086697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.087362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.088218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.167958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:47.169430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7eeb28-8455-4038-850e-2e87f545a5fa None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:47.285854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 160/316] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:47.292542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:47.468077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6856b9-723b-4d23-a478-a7cb8b13086a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.468266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6856b9-723b-4d23-a478-a7cb8b13086a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.468544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6856b9-723b-4d23-a478-a7cb8b13086a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.470212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 161/317] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:47.474438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365d5b4-6eed-4139-a8ef-60cf576223d8 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:47.475584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.475872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.476149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.494912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:47.495596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:47.496197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:47.517827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:47.557165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:47.558277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.558782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.559318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.561431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:47.564367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:47.567962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 157/318] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:47 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:47.575840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:47.593335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 162/319] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:47.610621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:47.729441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad5129-2aa6-4e79-b3d3-9146b719a83d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:47.770019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:47.770755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.771194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.771765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.776361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebbaf78-5edb-46b0-98c8-f3d5624d9f27 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:47.784331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e57e861b-d562-4fd1-924d-8cecea19009d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:47.789395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:47.790408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:47.792520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 163/320] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:47.795551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38112f8-3e43-4a06-8ff1-572fe7dbdfce None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.797344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38112f8-3e43-4a06-8ff1-572fe7dbdfce None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.797651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38112f8-3e43-4a06-8ff1-572fe7dbdfce None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.799360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 164/321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:47.804653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38112f8-3e43-4a06-8ff1-572fe7dbdfce None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:47.829252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:47.847897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:47.849740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.850387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.851593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.853477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99, user.id=32810df2fa0c4913b0c49e5518e06e00) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:47.854207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:47.858166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812ea1e-9921-43ac-bbb7-7e87790b34ef None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:47.879118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 158/322] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:21:47 2022] GET /identity/v3/role_assignments?user.id=32810df2fa0c4913b0c49e5518e06e00&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 608 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:47.879118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:47.879118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:47.879118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:47.894170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:47.895484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:47.952113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:47.996535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38112f8-3e43-4a06-8ff1-572fe7dbdfce None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.006102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 159/323] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:48.030233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.042463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.043120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.043402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.043680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.044901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:48.045210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:48.046983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:48.058802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 165/324] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:48.067597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.159666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:48.204045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abb3c5-01ec-4f0a-b100-2a720d367d74 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.208815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd53c16-527b-4696-91df-3f116cd51426 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.251209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.251924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.252201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.252470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] PATH_INFO: `/v3/users/61f45aee715343a5b40a3b3e011c1676` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.253677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] RBAC: Authorizing `identity:get_user(user_id=61f45aee715343a5b40a3b3e011c1676)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:48.255185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:48.257143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 166/325] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:48 2022] GET /identity/v3/users/61f45aee715343a5b40a3b3e011c1676 => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:48.263394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.275684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.276772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.277250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.277667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.279545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:48.282271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707358f-86d4-4c19-ac71-bfbcd8396da5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:48.290154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 160/326] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:21:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:48.292605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767812-a35a-4b3c-97e7-19b163ddd9b3 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.293217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767812-a35a-4b3c-97e7-19b163ddd9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.293217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44767812-a35a-4b3c-97e7-19b163ddd9b3 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.294498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 161/327] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:48.295494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44767812-a35a-4b3c-97e7-19b163ddd9b3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.334091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:48.381581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ee1852-86ec-4fc7-ba8b-16e4330a01df None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.408693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44767812-a35a-4b3c-97e7-19b163ddd9b3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.423996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.424891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.425356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.425890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] PATH_INFO: `/v3/system/users/61f45aee715343a5b40a3b3e011c1676/roles/3d18bd6319274c7c8f05348d36f5fd6c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.428372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=61f45aee715343a5b40a3b3e011c1676, role_id=3d18bd6319274c7c8f05348d36f5fd6c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:48.430509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e85d4-3b00-48f5-aff3-85050c31f709 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:48.442481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 167/328] 10.222.0.33 () {58 vars in 1334 bytes} [Mon Aug 8 13:21:48 2022] PUT /identity/v3/system/users/61f45aee715343a5b40a3b3e011c1676/roles/3d18bd6319274c7c8f05348d36f5fd6c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:21:48.444232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.444714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.445189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.460558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.461201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.461465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.461840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] PATH_INFO: `/v3/groups` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.462498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:48.463227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:48.465496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:48.472088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:48.475747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:48.487062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 162/329] 10.222.0.33 () {60 vars in 1134 bytes} [Mon Aug 8 13:21:48 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:21:48.577238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 168/330] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:48.601778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.741904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:48.752489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4ad1-c635-4c89-8e1b-dba290245576 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:48.796990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:48.797651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:48.797935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:48.798205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:48.811935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:48.817302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:48.819150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:48.820604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 163/331] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:48.825369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:48.979142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa067253-361b-4c07-97c9-6129d7f24938 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:49.040914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:49.041877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:49.042339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:49.042796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:49.044138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:49.044701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:49.047137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:49.051490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:49.065317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 169/332] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:49.071150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:49.270171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18480371-365f-4665-81cf-87cb79612ea3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:49.341715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:49.342871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:49.343200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:49.343671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] PATH_INFO: `/v3/users/b031076105584f259adc62ac84d3f48e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:49.345575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] RBAC: Authorizing `identity:get_user(user_id=b031076105584f259adc62ac84d3f48e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:49.348281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83deb238-8a3b-4ea4-a03d-3a8500aa9212 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:49.351656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 164/333] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:49 2022] GET /identity/v3/users/b031076105584f259adc62ac84d3f48e => generated 293 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:49.358250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:49.566962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd763b9c-0d7c-4645-955a-93a644c6a18c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:49.635775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:49.636758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:49.637220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:49.637688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:49.638924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] RBAC: Adding query filter params (role.id=2cb0880dbcea4e93aefe6370c68adecf, user.id=b031076105584f259adc62ac84d3f48e, scope.system=all) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:49.639492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:49.642277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3704337c-d72a-402a-9ecc-20b95b6a1d88 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:49.666706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 170/334] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 13:21:49 2022] GET /identity/v3/role_assignments?user.id=b031076105584f259adc62ac84d3f48e&scope.system=all&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 510 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:50.709772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f0a1-443e-49a7-a731-56f950443e84 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:50.710832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f0a1-443e-49a7-a731-56f950443e84 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:50.711365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1f0a1-443e-49a7-a731-56f950443e84 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:50.713453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 165/335] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:50.719855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:50.720822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:50.721296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:50.748367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:50.749978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:50.821053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:50.847822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 171/336] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:50.897434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1f0a1-443e-49a7-a731-56f950443e84 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:50.935624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:51.030022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1f0a1-443e-49a7-a731-56f950443e84 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:51.076388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:51.077066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.077347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.077620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.095171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:51.100575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:51.101536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:51.102910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 166/337] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:51.110658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:51.344999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc27633-90de-40e1-ae9b-ffc8a475f5c6 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:51.435024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:51.436002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.436470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.436941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.438269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:51.438817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:51.441544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:51.465254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 172/338] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:51.474343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:51.606342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:51.626581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaeb818-f8ca-4235-b8b1-2685116479b6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:51.671476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:51.672137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.672415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.672686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] PATH_INFO: `/v3/users/61f45aee715343a5b40a3b3e011c1676` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.673867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] RBAC: Authorizing `identity:get_user(user_id=61f45aee715343a5b40a3b3e011c1676)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:51.675387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fb5317-f2f2-4637-9dab-2556b285e3a0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:51.677459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 167/339] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:21:51 2022] GET /identity/v3/users/61f45aee715343a5b40a3b3e011c1676 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:51.687184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:51.688220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d130db-3bfb-41ae-b086-68f3b5a75af7 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.688506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d130db-3bfb-41ae-b086-68f3b5a75af7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.688777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d130db-3bfb-41ae-b086-68f3b5a75af7 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.690167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 168/340] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:21:51.705896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.706208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.706439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.733335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:51.735131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:51.819467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:51.878393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 169/341] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:51.906712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b235b-0a56-4606-a2a5-c5c7ee3a42a8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:51.912085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:51.997393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:51.998028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:51.998305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:51.998571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:51.999346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] RBAC: Adding query filter params (role.id=3d18bd6319274c7c8f05348d36f5fd6c, user.id=61f45aee715343a5b40a3b3e011c1676, scope.system=all) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:51.999657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:52.001204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:52.032735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 173/342] 10.222.0.33 () {56 vars in 1308 bytes} [Mon Aug 8 13:21:51 2022] GET /identity/v3/role_assignments?user.id=61f45aee715343a5b40a3b3e011c1676&scope.system=all&role.id=3d18bd6319274c7c8f05348d36f5fd6c => generated 510 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:52.130506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:52.182245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93949d8e-4020-4460-8582-a7d1036c581b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:52.262043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:52.263113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:52.263610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:52.264086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:52.294416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b2bcd81e-6b75-4ee0-b9e2-aaa199121f51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:52.307955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:52.308938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:52.309735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:52.310827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 170/343] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:52.319028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:52.370540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51a2c-2f98-4123-9f88-8b2d260ecf70 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:52.371041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51a2c-2f98-4123-9f88-8b2d260ecf70 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:52.371555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff51a2c-2f98-4123-9f88-8b2d260ecf70 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:52.374251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 171/344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:52.382266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:52.382266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:52.382266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:52.440948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:52.443044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:52.560508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fbc25-5e8a-47e1-a03c-9a1ea0cfd999 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:52.569721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:52.614003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:52.615060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:52.615600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:52.616091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:52.617310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:52.617881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:52.620551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:52.657523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 174/345] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:52.661399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:52.703818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 172/346] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:21:52.793384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:52.907416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e24362-acef-4aad-a6f9-c10f453846a7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:52.960564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:52.961115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:52.961617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:52.962433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:52.983761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.038744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.053620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a7644-6ac3-4460-a397-93ee10ecb37b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:53.115603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:53.117246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:53.136536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:53.137194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.137469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.137739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.172409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:53.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:53.179349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:53.180343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:53.182954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 173/347] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:21:53.188786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:53.239405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 175/348] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:53.248319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:53.341567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184d615-53d1-4555-b9d4-d5d2bc9bd924 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:53.351662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.397083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895e303-f513-48b5-a204-dcd2c2e00c68 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:53.436973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:53.438041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.438553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.439014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.441036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:53.443446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:53.445747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 176/349] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:53 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:53.451563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:53.462359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.489148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:53.490255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.490701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.491204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.492432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:53.493014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:53.497592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:53.537919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 174/350] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:53.550707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:53.707611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9405b-a8b9-441d-a60b-f5e9c0c2b2fd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:53.757416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:53.758570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.759060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.759562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.764990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.771160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:53.771443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bbb74-1478-4e1f-a10e-f22a2f01604d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:53.777061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28484e-41ad-4acc-b4d2-02e4b1709ba1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:53.783836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 177/351] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:53 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:53.787035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97e71-133a-4ffd-b9b4-ac16ac5e2981 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.787860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97e71-133a-4ffd-b9b4-ac16ac5e2981 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.788544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b97e71-133a-4ffd-b9b4-ac16ac5e2981 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.792962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 178/352] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:53.795841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de927a-f2e0-4532-b743-1c88b97d3cb7 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.796345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de927a-f2e0-4532-b743-1c88b97d3cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.796807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25de927a-f2e0-4532-b743-1c88b97d3cb7 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.799424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 179/353] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:53.803176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de927a-f2e0-4532-b743-1c88b97d3cb7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:53.818575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:53.820034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:53.820517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:53.821102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:53.930909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:53.941710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:53.943809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaa7442-d35c-45ca-b35f-ccf4987844bf None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:53.954737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25de927a-f2e0-4532-b743-1c88b97d3cb7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.012153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.012841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.013142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.013393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.015163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=6249ead7798a4166a3569d19a133c509) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:54.015502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:54.017007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414f79d3-cc4d-4e54-b43b-52d6a192aef1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:54.023546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 180/354] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:21:53 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:54.025038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.025373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.025659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.053389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:54.054776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:54.132647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 175/355] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:54.134410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.139728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.140174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.141846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:54.153790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 181/356] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:54.154887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.164428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:54.166248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:54.253990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 176/357] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:54.255175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.280652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fa588c-c529-441c-a312-fd53bf7e42f4 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.306696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:54.329357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.330257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.330366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.330594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.332964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:54.334424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087d82ea-e90c-4fd0-a8ff-b10b2bca9eeb None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:54.336681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 182/358] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:54.338050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc752a-3096-46a5-af0a-e03bcbe9ff52 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.338339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc752a-3096-46a5-af0a-e03bcbe9ff52 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.338614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dc752a-3096-46a5-af0a-e03bcbe9ff52 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.340075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 183/359] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:54.340945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc752a-3096-46a5-af0a-e03bcbe9ff52 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.431737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73fac6-bf6d-490c-b58f-b64399f93823 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.441787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:54.481043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.481705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.481982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.482255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.496676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:54.497996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dc752a-3096-46a5-af0a-e03bcbe9ff52 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:54.501976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:54.502951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:54.503793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:54.504521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 177/360] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:54.505369 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.546690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.547681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.548157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.548712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.562739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-120f75f6-a94e-4869-989f-93418a8f8896 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:54.568740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:54.569884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:54.570901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:54.573460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 184/361] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:54.575026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.575464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.575869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.596997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:54.598251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:54.633433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:54.662430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ace3275-82d7-4a75-97af-f008e099ccb6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.692895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 185/362] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:54.693825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.713965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.715119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.715658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.716293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.717949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:54.719792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:54.723242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 178/363] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:54.724629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.771375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:54.829000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e8eed-8e82-447b-9517-aaea74761356 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.877873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.878699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.879024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.879336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.880037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:54.880341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:54.881758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:54.881943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6af3f-856b-4634-8222-776c788192a1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:54.895598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 186/364] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:54.896707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.935466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:54.936091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:54.936356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:54.936630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:54.937322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:54.937620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:54.939601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:54.950704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 179/365] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:54.952115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:54.976860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.061838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e902560-cf0f-45e2-a975-1bf1ca752ed0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.086673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.116436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb69fbe-fdc4-470b-a6be-74df10c81263 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.138022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:55.140110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:55.141008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:55.141855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:55.167535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:55.179040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:55.180756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:21:55.181953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:55.182967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:55.184009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:55.184466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:55.187053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:55.187273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 187/366] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:55.187393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=6249ead7798a4166a3569d19a133c509) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:55.187505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:55.187625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:55.189460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:55.213970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 180/367] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:21:54 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 260 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:55.215846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:55.361850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2facd84c-009c-4963-ab9d-d17de8d2d2eb None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.380921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.416205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552fd51-6f30-4188-bb4a-144aca5f150f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.426823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:55.427529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:55.427811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:55.428087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:55.469115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:55.469767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:55.471030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:55.472072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:55.507519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.539862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:55.541419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:55.626361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:55.629583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:55.708832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 188/368] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:55.710301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:55.741931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.804954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 181/369] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:55.805913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:55.868037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:55.929149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8f9b-6f11-488d-86e3-370bb038ebad None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.950050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6809889b-c61e-475f-9b53-e564c81f62fd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:55.996851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:55.997514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:55.998326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:55.998609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.000283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.001776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.003986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 182/370] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:55 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.004858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.009021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.010463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.010945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.013071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.014686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:56.015275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.017973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.034111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 189/371] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.036938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.116908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.185668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19539e02-fc9a-4759-801c-9a5bdd22de53 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.208835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c028c50-78d0-4351-a3b3-2859cd9ea5ff None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.226347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.227036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.227347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.227619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.228817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.230271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.231944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 190/372] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.232747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.236766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.277204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.278338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.278803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.279320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.281389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.284207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.287537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 183/373] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.289126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.352556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c588f0-13d7-4315-a9c5-80316d1ef5e2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.405366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.406545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.407340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.408075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.460531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87896cf-3f1d-429b-98aa-17c04bb550c7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.479604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.501769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.503570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.503713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.504384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.504758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.505126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.505989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:56.506394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.507790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.508596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.516508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 184/374] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 260 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.517516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.699609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 191/375] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:21:56.705476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.705664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a47390-26a3-482f-8f12-86101a8a8b8a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.750323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298780-7171-402d-9974-42a71befd961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.750904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298780-7171-402d-9974-42a71befd961 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.751209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298780-7171-402d-9974-42a71befd961 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.751492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a298780-7171-402d-9974-42a71befd961 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.752805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a298780-7171-402d-9974-42a71befd961 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.754320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a298780-7171-402d-9974-42a71befd961 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.756453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 185/376] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:56.763184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298780-7171-402d-9974-42a71befd961 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:56.824091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.905942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99a7a6-3650-4a63-9a9b-d712ec37e62d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.908941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:56.925172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a298780-7171-402d-9974-42a71befd961 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:56.953899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:56.954941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.955472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.955940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] PATH_INFO: `/v3/projects/cd8972f6a1684e42be6181a89144c88c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.957383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] RBAC: Authorizing `identity:get_project(project_id=cd8972f6a1684e42be6181a89144c88c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:56.959222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f32da-5cd8-45a8-bf34-4674b2b5fe97 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:56.961672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 192/377] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/projects/cd8972f6a1684e42be6181a89144c88c => generated 344 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:56.963439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794c8c-c442-4e48-972f-7361bc6fed93 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:56.963927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794c8c-c442-4e48-972f-7361bc6fed93 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:56.964408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25794c8c-c442-4e48-972f-7361bc6fed93 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:56.966237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 193/378] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:56.971605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25794c8c-c442-4e48-972f-7361bc6fed93 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.009286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.010960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.011496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.011962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.012901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:57.013468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:57.015385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f5b2f5-c8b4-4d99-b056-d39bb071ff19 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:57.024189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 186/379] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 260 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:57.025841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.026325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.026876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.058704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:57.060381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:57.176196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25794c8c-c442-4e48-972f-7361bc6fed93 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:57.223252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.224051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.224254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.224531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] PATH_INFO: `/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.225773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 187/380] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:57.238187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] RBAC: Authorizing `identity:get_group(group_id=2b7234b971f642b2838ad12a0d6ed1e1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:57.240054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:57.242343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 194/381] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:56 2022] GET /identity/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1 => generated 229 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:57.248399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.259512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.313628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:57.339854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:57.466241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9ff732-1ebb-4195-8ee6-f8ab87fa522d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:57.528763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94e6efb-d5b3-4d7a-be4d-38333f96659b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:57.531444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.532533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.533019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.533596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.556033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:57.564780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:57.566980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:21:57.568990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 195/382] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:21:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:57.573561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.649940 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.650575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.650852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.651153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.651921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] RBAC: Adding query filter params (group.id=2b7234b971f642b2838ad12a0d6ed1e1, role.id=193a5d91676042449492edcffd7d1760, scope.project.id=cd8972f6a1684e42be6181a89144c88c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:57.652229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:57.653726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6469d-79f4-4f8c-a299-5efe0b4da3eb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:57.673232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 188/383] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:21:57 2022] GET /identity/v3/role_assignments?group.id=2b7234b971f642b2838ad12a0d6ed1e1&scope.project.id=cd8972f6a1684e42be6181a89144c88c&role.id=193a5d91676042449492edcffd7d1760 => generated 260 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:57.696678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:57.697830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c464e-29f4-44f8-ad64-7975c2d31497 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.698119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c464e-29f4-44f8-ad64-7975c2d31497 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.698392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c464e-29f4-44f8-ad64-7975c2d31497 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.700116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 189/384] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:57.701369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc49354-6d50-4479-bdb1-7c5067bdfee0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:57.706778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.707203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.707355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.744883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.745548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.745873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.746015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:57.746202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.746827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:57.747156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:57.748608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:57.755201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:57.759539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 196/385] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:21:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:57.767197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.893813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4509e748-54a5-4835-885c-fad99c212664 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:57.936403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:57.937204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:57.937526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:57.938119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:57.945478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:57.947478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:57.949844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 197/386] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:57 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:57.956629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:57.983597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 190/387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:57.995675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:58.013652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.058868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:58.092248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608d406c-cedf-4c45-8348-8dc21fdc8d0e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.133475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.134615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.135444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.135863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.137906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:58.139959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:58.142466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 198/388] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:57 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:21:58.149367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.236233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4938cf7a-10ff-4484-8e95-2c85440a162e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.280964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.281914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.282335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.282741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.296468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:21:58.302005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:21:58.303060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:21:58.304020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:21:58.306452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 191/389] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:21:58.312001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.360600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332a659-c3cf-4f58-bf88-229448cb8a96 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.373988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:58.452017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.452613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.452894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.453168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.460897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, group_id=44c47a817f85462d9af9e15d0bd2ee78, role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:58.462675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:58.480004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:58.490460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b6347b-f41e-49d5-839d-b6eb2138cc7b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.499820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 199/390] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 13:21:58 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:21:58.535402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.536040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.536318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.536591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.537290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:21:58.537594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:58.539040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:58.549328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 192/391] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:21:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:58.556587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.706376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7d0bad-f24a-4b01-9590-ef7b58910730 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.745892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.746576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.746853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.747152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.748521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:58.750028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:58.752098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 200/392] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:21:58 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:58.760698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.888398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbbe0ed-c83c-41d7-ad03-c0dd370ecd3d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:58.931262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:58.931909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:58.932189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:58.932455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:58.933631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:58.935123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29b86f-230a-45ea-b520-a28076418ec7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:58.937195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 193/393] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:21:58 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:21:58.942919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:58.972324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:21:59.092502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e139aa03-3065-466e-85ce-d8d258916bd4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:21:59.137681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:21:59.138398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:59.138670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:59.138935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:59.144955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=6249ead7798a4166a3569d19a133c509, group_id=44c47a817f85462d9af9e15d0bd2ee78, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:21:59.146578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29028bf-00e6-40cc-a18f-b053dad640d3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:21:59.176956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 201/394] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 13:21:58 2022] PUT /identity/v3/projects/6249ead7798a4166a3569d19a133c509/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:21:59.596899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8b66e-f472-4e00-98c2-b59c9876663c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:59.597918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8b66e-f472-4e00-98c2-b59c9876663c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:59.597918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb8b66e-f472-4e00-98c2-b59c9876663c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:59.600673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 194/395] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:21:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:21:59.609331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:21:59.610297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:21:59.610765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:21:59.639607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:21:59.641263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:21:59.749205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 202/396] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:21:59.784338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8b66e-f472-4e00-98c2-b59c9876663c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:21:59.850626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:00.016428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb8b66e-f472-4e00-98c2-b59c9876663c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:00.102394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:00.103705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.104203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.104680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.136702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-9dcc985c-5d13-4a21-82fc-a1e0691af68e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:00.147049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:00.148142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:00.148970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:00.149581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 195/397] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:21:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:00.158483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:00.187161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:00.372630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde118b-5618-4a0d-b3cb-8f7e46d1a808 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.373295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde118b-5618-4a0d-b3cb-8f7e46d1a808 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.373788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde118b-5618-4a0d-b3cb-8f7e46d1a808 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.375713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 196/398] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:00.387057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd04c-6b82-478f-b865-9b995b800602 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.387596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd04c-6b82-478f-b865-9b995b800602 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.388139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478dd04c-6b82-478f-b865-9b995b800602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.436044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-478dd04c-6b82-478f-b865-9b995b800602 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:00.438227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-478dd04c-6b82-478f-b865-9b995b800602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:00.447181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054943a4-2c44-42f8-8663-7a7c14563f19 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:00.561447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:00.562552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.563047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.563554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.564842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:00.565376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:00.568028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:00.601059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 203/399] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:22:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:00.604759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:00.647124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 197/400] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:00.650534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc3ba3-cd05-4e90-a920-882084e34a3c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.651285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc3ba3-cd05-4e90-a920-882084e34a3c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.651993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effc3ba3-cd05-4e90-a920-882084e34a3c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.654567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 198/401] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:00.663338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.664144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.664936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.697072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:00.699445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:00.719043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:00.758407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:00.811881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf48172-21ea-4f14-89f7-e953709760d3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:00.849907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 199/402] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:00.852561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:00.898042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:00.899138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:00.899620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:00.900091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:00.904398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:00.907678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:00.911762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 204/403] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:00 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:00.913742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.082323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d87b6-0d38-4f01-8eba-04de21229313 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.133586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0de4101-911e-4702-9522-c048b00e4de4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.153376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.154932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.155752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.156734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.179795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:01.190511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:01.192368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:01.193974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:01.199331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 200/404] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:01.201553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.208783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.209450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.209748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.209990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.213187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:01.223408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:01.229021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:01.230723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:01.232793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:01.235457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 205/405] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:22:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:01.236519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.239877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:01.369923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb48898f-381b-44b3-ad10-b982232ed816 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.407335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b72cd1-54ef-4b81-8be5-e2cc42745f87 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.434776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.435995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.436448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.436859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.438606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:01.440913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:01.442886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 201/406] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:01.443911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.478227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.479366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.479923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.480406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.481672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:01.482222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:01.484980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:01.501487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 206/407] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:01.506181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.546438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:01.596068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6fb0a-4fb8-4222-b55b-ddb9b17449cb None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.626463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:01.650677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.651952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.652401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.652809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.653781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:01.654236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:01.656820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:01.673254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 202/408] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:01.673254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.708034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633c04-43d7-4cd6-9319-2973afd74767 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.767998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.769733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.770208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.770644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.774249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, group_id=44c47a817f85462d9af9e15d0bd2ee78, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:01.776004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:01.795810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 207/409] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 13:22:01 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:01.797193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.824721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d208398-c5be-4a08-b6dd-d18da7e2a3a2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:01.872353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:01.873381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:01.875291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:01.875817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] PATH_INFO: `/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:01.877653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] RBAC: Authorizing `identity:get_group(group_id=2b7234b971f642b2838ad12a0d6ed1e1)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:01.881085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:01.885209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 203/410] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1 => generated 229 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:01.893193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:01.898664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:02.018870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57fb07d-8594-44b8-866a-095efe2d540c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.064170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:02.086537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.087800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.088335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.088858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.089434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fce91-aa81-44b1-a373-21f8824a558b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.090988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:02.093681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6abaab-5c52-4c89-8e6a-6b9155c19ce1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:02.101217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 208/411] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:02.101217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9a6a-56e0-498e-beca-b4470261dd1c None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.101217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9a6a-56e0-498e-beca-b4470261dd1c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.101217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12d9a6a-56e0-498e-beca-b4470261dd1c None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.102546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 209/412] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:02.105302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d9a6a-56e0-498e-beca-b4470261dd1c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:02.152122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.153176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.153696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.154182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] PATH_INFO: `/v3/projects/cd8972f6a1684e42be6181a89144c88c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.156495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] RBAC: Authorizing `identity:get_project(project_id=cd8972f6a1684e42be6181a89144c88c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:02.159325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc546e36-851a-4f5e-bac4-c9e36ac923c7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:02.163358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 204/413] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:01 2022] GET /identity/v3/projects/cd8972f6a1684e42be6181a89144c88c => generated 344 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:02.171110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.171110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.171110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.198513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:02.200306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:02.316523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12d9a6a-56e0-498e-beca-b4470261dd1c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.326175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:02.345399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 205/414] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:02.345399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:02.437509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.438557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.439051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.439540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.441828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:02.444692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:02.448356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 210/415] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:02 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:02.453908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:02.499942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3a9c7-94db-48d6-b4e1-70afe547fd00 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.552577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.554003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.554484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.555544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] PATH_INFO: `/v3/projects/cd8972f6a1684e42be6181a89144c88c/groups/2b7234b971f642b2838ad12a0d6ed1e1/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.558140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd8972f6a1684e42be6181a89144c88c, group_id=2b7234b971f642b2838ad12a0d6ed1e1, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:02.561105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:02.561398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:02.591226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 206/416] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 13:22:02 2022] PUT /identity/v3/projects/cd8972f6a1684e42be6181a89144c88c/groups/2b7234b971f642b2838ad12a0d6ed1e1/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:02.592628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:02.650094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f35e40-9ed6-4476-8444-4786a03dac4c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.736089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:02.753242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.753242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.753242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.753242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.766642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-3c34293b-5b1b-41a4-af72-0c387bd59607 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:02.772716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:02.774591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 211/417] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:22:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:02.775842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aab3e-91b7-4159-8588-2c4e1590142a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.777223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aab3e-91b7-4159-8588-2c4e1590142a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.777223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767aab3e-91b7-4159-8588-2c4e1590142a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.782277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 212/418] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:02.784615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.784977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.785376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.806611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:02.808211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:02.820229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2baafb-4dfe-40a6-ad99-ffc43b51b365 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:02.884333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:02.886032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:02.886680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:02.887399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:02.890654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e6ec63aeb89436b8ec6b96456157b99, group_id=44c47a817f85462d9af9e15d0bd2ee78, role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:02.892399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:02.897298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 213/419] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:02.898315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:02.910290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 207/420] 10.222.0.33 () {58 vars in 1442 bytes} [Mon Aug 8 13:22:02 2022] PUT /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99/groups/44c47a817f85462d9af9e15d0bd2ee78/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:02.952421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:03.008241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:03.097711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7687e-2748-4c10-899a-9f2d4e2f0fd0 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:03.145473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c73682-c2c0-4650-9067-87ee1eb06877 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:03.159793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:03.161014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:03.161716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:03.162195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:03.163220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:03.163815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:03.166301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:03.182292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 214/421] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:03.183423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:03.191448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:03.211271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:03.211931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:03.212209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:03.212480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:03.227195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:03.232406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:03.234540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:03.236261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 208/422] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:03.240127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:03.376890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6458e2-970c-4324-9e43-21714cec29b4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:03.392894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:03.418010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:03.418690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:03.419069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:03.419368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:03.443043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cb7bfc-1245-4ca0-86b3-50c8e38a0670 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:03.505942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:03.506579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:03.506852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:03.511114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:03.511114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] RBAC: Adding query filter params (name=member) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:03.511114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:03.514958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:03.532853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 209/423] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:22:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:03.538109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:03.580671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:03.582298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:03.623670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:03.738346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e508-53f4-4e62-adfd-4e3f7076ee5b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:03.787299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 215/424] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:03.793731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:03.812197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:03.812833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:03.813113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:03.813385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:03.814150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:03.899404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:03.904790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:03.963637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:04.010476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e5115-17e3-44a7-a33e-68589f197562 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.062029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.062776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.063061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.063381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.064550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.066059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.068190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 216/425] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:03 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:04.076772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:04.129342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 210/426] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:04.148898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:04.203504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:04.217332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768c124-d10e-4cb3-8da2-d07cbdb4caf0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.257708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.258413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.258695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.258969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.260917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.262330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.264401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 217/427] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:04 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:04.276241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:04.391213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63818b9c-3657-46d4-918f-c27b4aba7b06 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.436028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.436697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.436981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.437234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] PATH_INFO: `/v3/projects/6249ead7798a4166a3569d19a133c509` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.441366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] RBAC: Authorizing `identity:get_project(project_id=6249ead7798a4166a3569d19a133c509)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.442890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.451772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 211/428] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:04 2022] GET /identity/v3/projects/6249ead7798a4166a3569d19a133c509 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:04.454160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:04.485200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:04.494721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb1b3-1f57-4d11-ac08-50ab6567133e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.548000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.548525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.548803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.549072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.549824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=6249ead7798a4166a3569d19a133c509) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:04.550129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.552875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.568544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 218/429] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:22:04 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 611 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:04.699438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02b7c0-9fcf-4187-8ad3-a9256b986545 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.756195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.757211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.757211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.757445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.764004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.765622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700e5bfa-4d13-44f2-bc46-c4482a88fb68 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.767807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 212/430] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:04 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:04.773913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:04.793122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:04.899895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7adbba5-8dc2-4e84-871e-15c6f30d661a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:04.946441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:04.947106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:04.947391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:04.947663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:04.948419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=6249ead7798a4166a3569d19a133c509) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:04.948722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:04.950270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd768f-f192-4c3e-8b57-c69d76880862 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:04.957661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 219/431] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:22:04 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=6249ead7798a4166a3569d19a133c509&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:05.081374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd04ca-10b1-445b-a0b7-d91901103538 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.081707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd04ca-10b1-445b-a0b7-d91901103538 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.081923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecd04ca-10b1-445b-a0b7-d91901103538 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.083634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 213/432] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:05.089282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.089910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.090381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.111809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:05.113088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:05.197888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 220/433] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:05.222316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd04ca-10b1-445b-a0b7-d91901103538 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:05.319751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:05.346762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecd04ca-10b1-445b-a0b7-d91901103538 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:05.394284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:05.394284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.394284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.394284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.406794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-5d114dc4-178b-4b6a-af6c-682e94218270 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:05.411580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:05.412517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:05.413323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 214/434] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:05.419411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:05.428246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:05.528040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458514d6-1eda-4ffa-bd99-c0f323be281f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:05.528266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd759-a1fe-4f38-b065-5fd9737e0b6b None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.528266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd759-a1fe-4f38-b065-5fd9737e0b6b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.528486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47fd759-a1fe-4f38-b065-5fd9737e0b6b None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.530145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 215/435] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:22:05.536752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.537035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.537310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.575041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:05.575924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:05.576747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.577162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.578072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.578612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] RBAC: Adding query filter params (name=member) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:05.579036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:05.580640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:05.585472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:05.594084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 221/436] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:22:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:05.600038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:05.688986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 216/437] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:05.720911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:05.729654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:05.765488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d361f7-05eb-47cd-b36d-40a43f7ee950 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:05.807574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:05.808592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.808592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.808856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.815908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:05.862065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462676d-c546-4094-94e8-725cbf6ba7db None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:05.907690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:05.908681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:05.909134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:05.909652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:05.915474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:05.916954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:05.924187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:05.930794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:05.932524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:05.933441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 217/438] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:05.939628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.013337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.062077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 222/439] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:22:06.067996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.085099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823d9773-388f-4a93-837a-36cdfaa9fdd6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.121924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.138063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f3834-bf35-4999-8506-30764215d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.139236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3834-bf35-4999-8506-30764215d053 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.139803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3834-bf35-4999-8506-30764215d053 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.140359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f3834-bf35-4999-8506-30764215d053 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.141617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f3834-bf35-4999-8506-30764215d053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:06.142124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f3834-bf35-4999-8506-30764215d053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.144550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f3834-bf35-4999-8506-30764215d053 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.163309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 218/440] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:06.164945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f3834-bf35-4999-8506-30764215d053 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.217926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9800a-08f2-4326-8e52-e62134f4d082 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.268836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.269497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.269778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.270060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.271709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.273201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.275021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 223/441] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:06.281420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.311420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f3834-bf35-4999-8506-30764215d053 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.329048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.366220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.367214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.367700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.368024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.439109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3072ec-74e5-44e1-bf6a-d2d133a2ce94 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.463887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.478702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.480527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.490191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.491234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.491623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.492293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.494071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.496012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2caaf4a-538e-4181-bc28-1851443431dc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.497937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 224/442] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:06.499586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650697d3-68ed-4122-8d31-e28bc1304c0e None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.500042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650697d3-68ed-4122-8d31-e28bc1304c0e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.500442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650697d3-68ed-4122-8d31-e28bc1304c0e None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.503118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 225/443] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:06.507836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.508288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.508620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.529913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:06.531535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:06.595162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 219/444] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:06.595947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.633805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 226/445] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:06.635705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.686700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.715645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:06.731743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f40cf-48a0-49f5-8ced-90fe6e184fdb None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.785990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.786932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.787408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.787845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.789112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=2cb0880dbcea4e93aefe6370c68adecf, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:06.789626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.791292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.792825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb111ea-42df-4c3e-840d-c988b71d74a8 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:06.800510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 220/446] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=2cb0880dbcea4e93aefe6370c68adecf => generated 611 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:06.801869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.858166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:06.859742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:06.860249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:06.860694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] PATH_INFO: `/v3/projects/cd8972f6a1684e42be6181a89144c88c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:06.862723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] RBAC: Authorizing `identity:get_project(project_id=cd8972f6a1684e42be6181a89144c88c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:06.866114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:06.877373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 227/447] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/projects/cd8972f6a1684e42be6181a89144c88c => generated 344 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:06.879427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:06.974007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f15dd2d-2fa8-4709-803d-6abcc8e57f55 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.027108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.029437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.029437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.029772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.046408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:07.054714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:07.056547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:07.058039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 221/448] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:07.062922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32702d7-28f7-46bd-baff-2b310374dfe7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.065992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:07.069262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:07.096816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:07.126360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.127336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.127797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.128196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] PATH_INFO: `/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.129905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] RBAC: Authorizing `identity:get_group(group_id=2b7234b971f642b2838ad12a0d6ed1e1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:07.132489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:07.139164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 228/449] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:06 2022] GET /identity/v3/groups/2b7234b971f642b2838ad12a0d6ed1e1 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:07.146119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:07.246307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e05a95-4617-4c57-a020-7903c469746d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.306468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.307396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.307782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.308113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.308866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:07.309227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:07.310886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:07.321862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993232a1-d8f1-427d-8f33-609f2da44bb8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.322113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 222/450] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:07.328631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:07.361450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.362079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.362356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.362627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.363424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] RBAC: Adding query filter params (group.id=2b7234b971f642b2838ad12a0d6ed1e1, role.id=193a5d91676042449492edcffd7d1760, scope.project.id=cd8972f6a1684e42be6181a89144c88c) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:07.363735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:07.365246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:07.374073 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 229/451] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:22:07 2022] GET /identity/v3/role_assignments?group.id=2b7234b971f642b2838ad12a0d6ed1e1&scope.project.id=cd8972f6a1684e42be6181a89144c88c&role.id=193a5d91676042449492edcffd7d1760 => generated 611 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:07.438324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:07.469833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad141f-a45a-4cdd-92cf-df1d6ff3fc7f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.511997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.512754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.513119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.513472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.605683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:07.607410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:07.690970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:07.717018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 223/452] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:22:07.723200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:07.761926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:07.835174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d9216-7505-4534-bf46-dc79ad66b1ab None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:07.875713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:07.876486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:07.876613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:07.876891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] PATH_INFO: `/v3/projects/4e6ec63aeb89436b8ec6b96456157b99` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:07.878144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] RBAC: Authorizing `identity:get_project(project_id=4e6ec63aeb89436b8ec6b96456157b99)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:07.879651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:07.881309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 230/453] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:22:07 2022] GET /identity/v3/projects/4e6ec63aeb89436b8ec6b96456157b99 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:07.887916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:07.994473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3de67-b694-4d66-9973-62eda7d4291d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:08.032944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:08.033591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:08.033868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:08.034140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] PATH_INFO: `/v3/groups/44c47a817f85462d9af9e15d0bd2ee78` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:08.035344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] RBAC: Authorizing `identity:get_group(group_id=44c47a817f85462d9af9e15d0bd2ee78)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:08.036792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616599d-0ec4-4295-b490-96a811aee41a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:08.038410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 224/454] 10.222.0.33 () {56 vars in 1180 bytes} [Mon Aug 8 13:22:07 2022] GET /identity/v3/groups/44c47a817f85462d9af9e15d0bd2ee78 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:08.044892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:08.156298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4119003-a422-44e0-a0cc-946273fd7b47 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:08.157969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:08.194796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:08.195421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:08.195776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:08.196076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:08.196820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] RBAC: Adding query filter params (group.id=44c47a817f85462d9af9e15d0bd2ee78, role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=4e6ec63aeb89436b8ec6b96456157b99) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:08.197126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:08.198609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70838d64-df4a-43d0-9d54-a2d61ca29d2d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:08.205439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 231/455] 10.222.0.33 () {56 vars in 1376 bytes} [Mon Aug 8 13:22:08 2022] GET /identity/v3/role_assignments?group.id=44c47a817f85462d9af9e15d0bd2ee78&scope.project.id=4e6ec63aeb89436b8ec6b96456157b99&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:11.271927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d66d-7678-439c-b8f7-864a343ba788 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:11.272794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d66d-7678-439c-b8f7-864a343ba788 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:11.272794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8d66d-7678-439c-b8f7-864a343ba788 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:11.274246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 225/456] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:11.282126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:11.282890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:11.282890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:11.312289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:11.313526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:11.420434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 232/457] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:11.484735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8d66d-7678-439c-b8f7-864a343ba788 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:11.544257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:11.686208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8d66d-7678-439c-b8f7-864a343ba788 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:11.771622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:11.772693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:11.773156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:11.773690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:11.799723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:11.801671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7161f8d3-f3fb-448a-9148-d63d6c28823e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:11.822608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:11.823620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:11.824463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:11.824463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:11.824463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:11.826398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 226/458] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:11.836355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:11.891597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f9d3-7abc-47f2-9ca0-93eb0f4db230 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:11.892551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f9d3-7abc-47f2-9ca0-93eb0f4db230 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:11.893030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9f9d3-7abc-47f2-9ca0-93eb0f4db230 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:11.895617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 227/459] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:11.909485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c132e-b063-44b4-8c06-ad99b543dba3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:11.910109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c132e-b063-44b4-8c06-ad99b543dba3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:11.910582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c132e-b063-44b4-8c06-ad99b543dba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:11.946846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-130c132e-b063-44b4-8c06-ad99b543dba3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:11.950246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-130c132e-b063-44b4-8c06-ad99b543dba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:12.026914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.034652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0780cf-9cc0-4f91-b147-39a29e706273 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:12.074502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:12.075550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.076033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.076519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.077433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:12.077937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:12.078180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 228/460] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:12.079835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:12.080759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42555b9-fdeb-4d20-a068-11c9d83742c7 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.081254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42555b9-fdeb-4d20-a068-11c9d83742c7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.081925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42555b9-fdeb-4d20-a068-11c9d83742c7 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.091270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 229/461] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:12.093979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.094485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.094963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.095993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 233/462] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:12.104179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.145234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:12.147411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:12.186099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.271732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d01ce2-a3cd-448d-86d5-cfbf7dbae4a7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:12.278539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 230/463] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:12.285647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.326577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:12.328740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.329478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.330164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.331972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:12.334851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1507df-5b5c-42ef-8859-2436b6853101 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:12.374565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 234/464] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:12 2022] POST /identity/v3/users => generated 248 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:12.377412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095b201-f599-4e18-9cd6-07802ed22d06 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.378306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095b201-f599-4e18-9cd6-07802ed22d06 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.379844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c095b201-f599-4e18-9cd6-07802ed22d06 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.383428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 235/465] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:12.389081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095b201-f599-4e18-9cd6-07802ed22d06 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.431191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.451430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabcc5a-c708-4cf2-b546-d1461e7f22c3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:12.496335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:12.497080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.497395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.497638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.510477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:12.512038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b60951f8-cd8c-48d6-85dd-449a313114fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:12.530324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:12.532090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 231/466] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.534969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.553255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:12.554508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:12.557076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.634295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 232/467] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:12.642118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.645353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c095b201-f599-4e18-9cd6-07802ed22d06 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:12.713096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:12.714102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.714595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.715056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.736121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:12.738930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:12.769013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:12.770797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:12.772975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:12.772975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:12.772975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:12.772975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 236/468] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:12.773420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.827160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b62bb0-172c-428c-b31c-75890f47a5b3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:12.876174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:12.876814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:12.877090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:12.878152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:12.878911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:12.879323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:12.881190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:12.891443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.899938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 233/469] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:12.900930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:12.911776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:12.972539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58949166-600d-4030-9b6e-b43ada09ad9b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.037735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.038520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f4617-5942-49da-ae28-e6d66d863fbd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.038832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.039502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.039985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.059924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.062806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.083048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.083705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.083980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.084252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.084994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:13.085301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.086760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:13.089001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:13.091014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:13.092683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:13.092683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:13.092683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:13.092683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 237/470] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:13.092683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:13.110271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 234/471] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:13.111316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:13.189407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:13.246217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d6dcb0-afb8-467f-8427-0e3eb143cdd1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.264041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699ed33c-4d25-4e6d-b478-2dcd967ab29a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.292960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.294266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.294693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.295331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.296156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:13.296607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.298717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.319303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 235/472] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:13.319501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:13.319578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:13.333848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.335354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.335480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.335949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.337225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.340156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.391621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 238/473] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:13 2022] POST /identity/v3/users => generated 246 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:13.392467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:13.468850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d8095-c6dc-4a26-87e6-7e59f8b69a03 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.490268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:13.539178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.540626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.540887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.541111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.541783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.547352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a22b6d1-53dc-4625-a6e1-4c33f95f313f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.595307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 236/474] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:13 2022] POST /identity/v3/users => generated 249 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:13.614661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a013a9-ad2a-4ca1-83a6-4cf8126dd001 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:13.658692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:13.659377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.659657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.659934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.660624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:13.662141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68666f90-a6bd-4fa5-af7f-a045b352203a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:13.698606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 239/475] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:13 2022] POST /identity/v3/users => generated 248 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:13.953488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4578ee-16f4-4aab-94f2-e1a2b08b2a04 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.953780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4578ee-16f4-4aab-94f2-e1a2b08b2a04 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.954152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4578ee-16f4-4aab-94f2-e1a2b08b2a04 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:13.955931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 237/476] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:22:13.965702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:13.966191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:13.966680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:14.005623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:14.007863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:14.054429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:14.143050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 240/477] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:14.212342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4578ee-16f4-4aab-94f2-e1a2b08b2a04 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:14.361786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4578ee-16f4-4aab-94f2-e1a2b08b2a04 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:14.418343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:14.418982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:14.419288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:14.419560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:14.420272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:14.421796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:14.442959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.sql.core [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 13:22:14.442959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 13:22:14.442959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [parameters: {'id': '9f1e7c9423974100bd77b95ba367e64c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 13:22:14.442959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80188) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result.read() Aug 08 13:22:14.460109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return ref.to_dict() Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:22:14.461695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application yield resource Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise self.value Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application session.commit() Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 13:22:14.463467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.session.flush() Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application compat.raise_( Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 13:22:14.465103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:22:14.466783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result.read() Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:22:14.474974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [parameters: {'id': '9f1e7c9423974100bd77b95ba367e64c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:14.478075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 08 13:22:14.480817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:14.483543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 238/478] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 13:22:14 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:14.483731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:14.723385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb13bc-d518-4eba-95ac-aeab5756b3e2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:14.797519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:14.798179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:14.798454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:14.798735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:14.816161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-3bc89ade-7635-4eb2-aa83-0831848e66b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:14.821044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:14.822647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:14.824181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 241/479] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:22:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:14.831932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:14.933462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:15.074860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31bfda0-5e39-4ae5-91de-8f4c7e179d5a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:15.148307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:15.149312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:15.149759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:15.150195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:15.151400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:15.151917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:15.154403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e85b-213d-4bf1-b245-1b1257590181 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:15.174859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 239/480] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:15.898880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890b02-fa8c-46d5-9268-5ec6ef68ab16 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:15.899964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890b02-fa8c-46d5-9268-5ec6ef68ab16 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:15.899964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d890b02-fa8c-46d5-9268-5ec6ef68ab16 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:15.902774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 242/481] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:15.911705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:15.912663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:15.912663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:15.949533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:15.952001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:16.128292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 240/482] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:16.201446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d890b02-fa8c-46d5-9268-5ec6ef68ab16 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:16.248995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:16.475452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d890b02-fa8c-46d5-9268-5ec6ef68ab16 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:16.564821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:16.566410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:16.567070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:16.567760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:16.590944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-9f0ef41d-1a7a-4188-ba3c-306b2f2916ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:16.596597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:16.597623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:16.598453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:16.601102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 243/483] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:16.611107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:16.705866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:16.854057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bafa-3d28-45f1-b4dd-84bbd661f518 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:16.854756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bafa-3d28-45f1-b4dd-84bbd661f518 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:16.855383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7bafa-3d28-45f1-b4dd-84bbd661f518 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:16.857412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 244/484] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:22:16.863928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:16.864772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:16.865343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:16.880188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e457a-6e8c-4240-bbdd-d0e148fbdb69 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:16.886263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:16.888811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:16.961642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:16.963755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:16.964410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:16.965039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:16.966335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:16.967030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:16.969835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:16.991249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 241/485] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:16.996292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.033092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 245/486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:17.073917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.095710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.132367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.143314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b1121-355b-4e79-949f-61e8d3379d9e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.220143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.220796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.221070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.221339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.245430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:17.246918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:17.275344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:17.276326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:17.277163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:17.277163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:17.277163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:17.277163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 242/487] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:17.288889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.315303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929cf62c-692e-4ab6-9249-01171b67f736 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.364560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.365618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.366082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.366783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.380016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-0811226c-a33e-417c-8817-7ff9686523cb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:17.385995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:17.387226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:17.388301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:17.390527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 246/488] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:17.397264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389f7-559c-4653-85dd-840d2eaa5571 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.397799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389f7-559c-4653-85dd-840d2eaa5571 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.398231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db389f7-559c-4653-85dd-840d2eaa5571 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.400256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 247/489] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:17 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:17.401227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db389f7-559c-4653-85dd-840d2eaa5571 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.432141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f7cdc-0845-4bee-977c-ae307eb508df None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.472374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.473152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.473428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.473875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.474925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:17.475268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:17.477286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a526cecb-bf04-44df-a913-0dfe14e949b3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:17.482766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.494224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 243/490] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:17.495611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d6c4-316c-4d31-b703-24e23b45add2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.496073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d6c4-316c-4d31-b703-24e23b45add2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.496491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88d6c4-316c-4d31-b703-24e23b45add2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.498420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 244/491] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:17.500734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.501207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.501610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.510879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.523368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:17.524936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:17.551518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db389f7-559c-4653-85dd-840d2eaa5571 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.615128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 245/492] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:17.615128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.617853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.618516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.618797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.619059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.619969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:17.620432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:17.622006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca2703-07b4-4185-99ce-c31d22d41e3c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:17.632385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 248/493] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:17.634773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.635654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.635654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:17.655811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:17.748431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 249/494] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:17.749553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.779200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc0deb7-e8ad-45c8-8a0c-159dcbc07351 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.827486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.828500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.829021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.829464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] PATH_INFO: `/v3/users/glance` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.832766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.840854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:17.842762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:17.851058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:17.859613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:17.860901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:17.861952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:17.863912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 246/495] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:17.865110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:17.921513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0779ce-ceb1-4762-bd4e-26c83126db40 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:17.974749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:17.975773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:17.976200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:17.976785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:17.995029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:18.003221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:18.004264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:18.006287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 250/496] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:18.006969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.029361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd9b6f4-fd9a-498f-a128-8f8c03025dac None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.075136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.076061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.076483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.076908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.092012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.094255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:18.113485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:18.115328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:18.116966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:18.116966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:18.116966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:18.116966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 247/497] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:18.116966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.136494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a462-687f-44c9-9097-e46f35272586 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.162110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.177445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.177980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.178259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.178526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.186727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.192293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:18.197476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:18.198447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 251/498] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:18.199880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.274459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7c5b23-8f52-4df8-9625-904174658bcc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.324618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.325570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.325747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.326000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.326742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:18.327211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.328682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.342953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d6843-0827-4a55-9fd2-2af99372a09b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.379130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 248/499] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:18.379130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.396751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.397701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.398083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.398530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.399705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:18.400139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.402051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.419101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 252/500] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:18.419101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.468541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.521600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c8a3fa-c08d-447e-b17f-a190ad5b63f5 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.524528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.533948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86edf52c-36cc-4b72-b68c-8dca3b6a8874 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.563806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.564578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.564967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.565334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.566059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:18.566453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.568272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.572998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.573621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.573896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.574161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.574838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:18.575244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.576676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.583531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 249/501] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:18.584966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.586483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 253/502] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:18.587200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.695139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037a1cf-ddd7-40e6-80d7-e23256a1853a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.697981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.726638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f6515-333a-4d99-93d7-20204bd6878b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.737040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.737839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.738267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.738586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.751392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.753319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.769416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.770242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.770615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-8cc63a8b-27d5-4aef-a468-02c5f93e55d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:18.770921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:18.772388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:18.774082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:18.774082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:18.774082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:18.774435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.774582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 254/503] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:18.774582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1cb6-eeed-449b-b989-5278ea940c17 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.774582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1cb6-eeed-449b-b989-5278ea940c17 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.774582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d1cb6-eeed-449b-b989-5278ea940c17 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.781129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 255/504] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:18.781129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1cb6-eeed-449b-b989-5278ea940c17 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:18.816944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:18.864099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.865643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:18.928423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d1cb6-eeed-449b-b989-5278ea940c17 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:18.983160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:18.983914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:18.984327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:18.984724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:18.996619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:18.999012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:19.021585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:19.022664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:19.030963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 256/505] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:19.032127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 250/506] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:19.032558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.032639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.057388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.114120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.160490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83777b99-d95a-464a-872d-b762b59de6ff None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.200397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.201098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.201443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.201777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.202504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:19.202879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.203314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c8360-4272-4774-bc84-efc85bf49f50 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.204480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1abf6b5-2cb6-4067-87b9-de71d4f674bc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.217420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 251/507] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:19.218475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.218762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.219038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.238610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:19.239783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:19.254242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.254911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.255233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.255503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.268489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.270020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:19.286222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:19.288206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:19.289990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:19.289990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.289990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:19.291600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 257/508] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:19.293036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.324115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 252/509] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:19.325531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.392368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.416915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.446237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f06358-3d1b-4f92-8cd8-6899eabe0239 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.467454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547ef2f-3536-4646-b96f-45e4ab3856e1 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.501080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.502059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.502546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.502969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.503438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.504063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.504343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.504617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.505364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:19.505670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.507166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.523230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.526109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.526268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 253/510] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:19.527160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:19.550505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:19.551745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:19.552787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:19.552787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.552787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:19.552787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 258/511] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:19.554045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.665211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d59e7-61da-4a9c-8909-9d80ed6be8f4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.680910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.694015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349518d9-db36-4894-89a0-bfc52be250fe None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.706462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.707611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.708058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.708539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.716085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.721876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.723388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.732232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.732849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.733120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.733393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.734051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:19.734349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.735767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:19.735897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:19.736839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 254/512] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:19.737804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.754520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 259/513] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:19.755361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.853286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e707430-73fb-43ca-b431-212b25eb13e0 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.874668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e385528-90f6-4b9c-8626-9acf4aa552a9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:19.894536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.895280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.895572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.895990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.910700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:19.912530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:19.915352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:19.916260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:19.916622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:19.916958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:19.928337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:19.929332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:19.930247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:19.930247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:19.930247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:19.930247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 255/514] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:19.930477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:19.966355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:19.978463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.002030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.003555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.059966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711779ab-49ee-4243-8583-954dc1d2c7b4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.112835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 260/515] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:20.113086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.113495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.113743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.113886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.113998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.114750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:20.115057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.116506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.129848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 256/516] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:20.130717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.220890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.238872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99826d1-79d6-44b1-8022-df83c86f67e2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.249146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.269601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f9597b-1fb1-4a9b-964a-c6ccaba4f12c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.282614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.283303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.283587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.283857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.292100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.293586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:20.306410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:20.307849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 257/517] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:20.309530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.310796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.310796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.310796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.310796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.311309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:20.311309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.312118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.352935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 261/518] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:20.354711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.435849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958d0703-c8e6-449c-8fbb-dad103a480d6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.477887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.478968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.479452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.479839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.480924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:20.481394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.483839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.483994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507141f-83e2-494d-902f-24dcc9bbdf22 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.500910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 258/519] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:20.501854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.507185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.534724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.535404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.535681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.535950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.543272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.548044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.549567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:20.565377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:20.566480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:20.568234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:20.568234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:20.568234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:20.568234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 262/520] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:20.569503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.616865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aa55fd-42f2-45e1-980a-44f53c9af41e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.675616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.676674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.677150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.677619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.682794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057f4ce-1239-44cc-a99b-9cc4c9122c2c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.689534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.692379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:20.714145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:20.715957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:20.717854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 259/521] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:20.718935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.719641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.720437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.720844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.721227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.722133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:20.722510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.724247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.742713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 263/522] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:20.743593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.808021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.822165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:20.856913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e97c7-144a-4044-929f-e3f781871cc2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.880991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eab8b2-7ede-4797-93aa-0b9eb287d0f2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:20.895451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.896202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.896476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.896857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.919887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:20.920524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:20.920878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:20.921082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:20.921750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:20.923322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:20.950011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 264/523] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:20 2022] POST /identity/v3/users => generated 247 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:20.950893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:20.999612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.001130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.087194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.115429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c269ac-f461-4455-9394-91559cbd4dcf None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.119628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.142051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 260/524] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:21.142935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.181535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.182556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.182929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.183242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.183938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:21.184239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.185710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.207595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 265/525] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:21.212448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.320920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e20ad-2bbc-4c5d-923b-24307186a7de None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.345373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae341-b3c1-46c1-ab28-e9d9d1aae1ae None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.368051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.368684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.368959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.369247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.369950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:21.370251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.374873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.403128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 261/526] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:21.403128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.412775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.414255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.414255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.414255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.445201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.454783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.502495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.504081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.565031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7615eb2-1af0-4ee3-a8bf-fce3bcc73ee7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.610021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.610565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.611145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.611420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.624549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.626049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:21.643352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:21.644705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:21.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:21.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:21.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:21.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 262/527] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:21.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.653458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 266/528] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:21.654921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.748459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.760078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:21.801822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2272cd7-cd26-44f9-9ca3-f2c37854f1c4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.812231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9085cb-9fc7-4acd-8cb5-922878ed107a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:21.857320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.858600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.859287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.859988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.867061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:21.867989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:21.868432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:21.868871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:21.870175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=f034fd79c9ef454498ee0874d2c97d16) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:21.870683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.872644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:21.873153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.875126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:21.881158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 267/529] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/role_assignments?user.id=f034fd79c9ef454498ee0874d2c97d16&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:21.882273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:21.888549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:21.889876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:21.890851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:21.895053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 263/530] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:21.896037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.024386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22334da3-76c2-40c3-b2ed-fd91eaf6637e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.040975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535eb7a9-6675-43fd-9fa0-715b5cfee78b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.066353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.077060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.077683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.077956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.078227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.078985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.079332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.080778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.082068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.082723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.083002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.083308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.086286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.095010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 268/531] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:22.095930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.100709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.102273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:22.117893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:22.118830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:22.120039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:22.120039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:22.120039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:22.123563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 264/532] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:22.124504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.228320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea11f18e-db9e-4fb1-9444-235dd0db68b9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.238928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b460e-4fe6-4592-873d-35bf7c640ea2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.267424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.268057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.268513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.268786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.269505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.269810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.271412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.279977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.280782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.281154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.281525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.289886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.291295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 269/533] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:22.292175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.298137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:22.311977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:22.313130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:22.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:22.319249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 265/534] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:22.320466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.347620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.410386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21353f90-b7fc-47df-a5df-1662feb7691c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.430821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.434970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e292edf-42df-42e2-9eb8-f0417f7122bd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.449031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.449652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.450024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.450296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.451051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.451401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.452803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.465602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 270/535] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:22.466466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.477395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.478081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.478367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.478640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.479520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=656cb98cf9b147f993f2cd31ec4ea55c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.479836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.481413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.487749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 266/536] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/role_assignments?user.id=656cb98cf9b147f993f2cd31ec4ea55c&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:22.496434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.580965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6775344d-f338-43fb-83dd-0fd5011ae0b6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.605089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.672952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.673594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.673871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.674138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.674851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.675185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.677093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.703316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cccff4-cc5c-4fc6-a905-fd0028e08019 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.706118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 271/537] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:22.713199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.734005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:22.782192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.782449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.782614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.782895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.800550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.802077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:22.832962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:22.833885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:22.834801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 267/538] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:22.842234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:22.861622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229c848f-93eb-4987-a32b-a1ebce7b65ec None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:22.900423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:22.901357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:22.901837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:22.902329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:22.903284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:22.903793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:22.905447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:22.912333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 272/539] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:23.038296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d057d77e-c998-4166-b046-7d0b0abf4b28 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:23.129389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:23.131702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:23.132471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:23.133294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:23.135319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:23.136129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:23.139142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd454406-9129-4c83-b8d5-c200ed0f2499 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:23.175867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 268/540] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:23.185899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:23.272442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:23.357700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3cd7d4-ef72-40d4-ac14-b66b481ab384 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:23.408489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:23.409133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:23.409410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:23.409683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:23.410440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=a36d4232444f46c08f950d19ae395643) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:23.410743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:23.412246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489035aa-17c1-47ed-bb4b-501d63ea24b4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:23.423514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 273/541] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:23 2022] GET /identity/v3/role_assignments?user.id=a36d4232444f46c08f950d19ae395643&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:23.578610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5822-6217-4148-b052-14b2c509f75d None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:23.579122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5822-6217-4148-b052-14b2c509f75d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:23.579630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce5822-6217-4148-b052-14b2c509f75d None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:23.582614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 269/542] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:23.591200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:23.591494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:23.591788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:23.620421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:23.621589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:23.723165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 274/543] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:23.776950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce5822-6217-4148-b052-14b2c509f75d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:23.860588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:24.036182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce5822-6217-4148-b052-14b2c509f75d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:24.105143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:24.106198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.106646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.107111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.132212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ec9543e4-958a-462d-a997-49e21dcc0e11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:24.146106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:24.147864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:24.151214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 270/544] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:24.160510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:24.234132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:24.264438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bea0c-58b0-4cd8-bba8-b2858a535e13 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.264878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bea0c-58b0-4cd8-bba8-b2858a535e13 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.265339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bea0c-58b0-4cd8-bba8-b2858a535e13 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.268335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 271/545] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:24.277834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.278281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.278759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.323430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:24.325523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:24.417362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a0aba8-129a-4a16-b073-0b445ba3561c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:24.531968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:24.533480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.534237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.535012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.536530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:24.537340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:24.538657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 272/546] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:24.540771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:24.556241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 275/547] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:24.564757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:24.569414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:24.625458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:24.865455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283def7-328d-4900-93d2-28898a9b06d0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:24.866449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eeb1df-ca41-439d-b52a-cc2840a68241 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:24.871278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:24.937670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:24.938787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.939975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.940449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.961917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:24.964722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:24.967583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:24.968262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:24.968542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:24.968818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:24.995522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:24.997492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:24.999538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:24.999538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:24.999538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:25.002909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 276/548] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:25.010047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:25.013328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:25.015608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:25.018262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:25.020368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:25.023581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 273/549] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 460 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:25.040772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.158319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdba4b8-d728-40b3-b8cc-dff5a16d2a3b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:25.194918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:25.195526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.195989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.196270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.196934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:25.197238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:25.198653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:25.211404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 277/550] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:25.212356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:25.223943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.254148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:25.288991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765cc40-e809-460f-84e8-2e08b200786b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:25.391160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cce3f3-9314-4762-9b65-870a851fb512 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:25.418474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:25.419566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.420063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.420540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.421763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:25.422310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:25.424881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:25.445235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:25.446211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.446703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.447177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 274/551] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:25.447460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] PATH_INFO: `/v3/users/swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.453926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.469574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:25.472823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:25.519113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:25.520826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:25.524584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:25.528445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 278/552] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:25.539719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.682722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cea9ee-9a1d-4906-926e-9abfbf42f6bd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:25.684333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:25.755768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259c322-5686-484a-9c33-2501fd69cbbf None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:25.765792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:25.766872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.767402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.767885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.789072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:25.791968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:25.796969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:25.818570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:25.819233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.819570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.819853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.820766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:25.821084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-6876e4c2-ec82-4f3d-b84f-13044c95848d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:25.822367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:25.823355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:25.824201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:25.824201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:25.824201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:25.824367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d631f63d-362c-4924-b81f-cb8c78395f20 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:25.829185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 275/553] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:25.829185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fa7bd-9223-4033-9d7e-505b0cc8dc32 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.829185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fa7bd-9223-4033-9d7e-505b0cc8dc32 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.829185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08fa7bd-9223-4033-9d7e-505b0cc8dc32 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.831891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 276/554] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:25.833448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fa7bd-9223-4033-9d7e-505b0cc8dc32 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.847403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 279/555] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:25.848732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f02a2-b8c7-4b6f-8f95-119aec285d9b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.849021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f02a2-b8c7-4b6f-8f95-119aec285d9b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.849305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f02a2-b8c7-4b6f-8f95-119aec285d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.868883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-245f02a2-b8c7-4b6f-8f95-119aec285d9b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:25.870298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-245f02a2-b8c7-4b6f-8f95-119aec285d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:25.948328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 280/556] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:25.957557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051d145-81fc-4f4e-9fb6-81f457bb44a3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:25.958507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051d145-81fc-4f4e-9fb6-81f457bb44a3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:25.959245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051d145-81fc-4f4e-9fb6-81f457bb44a3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:25.967863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 281/557] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:25 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:25.970162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051d145-81fc-4f4e-9fb6-81f457bb44a3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:25.997959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08fa7bd-9223-4033-9d7e-505b0cc8dc32 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.044743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.050752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.051684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.052154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.052555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.053381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:26.053860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.056701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:26.077601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 277/558] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:26.077601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.142625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051d145-81fc-4f4e-9fb6-81f457bb44a3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.143239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.206219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.207068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.207444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.207716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.208059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38914dc2-0d76-41cb-b5c2-649e812bb100 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.263724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.264428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.264647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.264893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.276984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-69fa21c1-c73b-4600-9473-4f7455bf0314 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:26.281699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:26.283534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 278/559] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:26.284757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.285522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.285522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.306113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:26.307259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:26.328999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.331825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:26.401525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 279/560] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:26.402772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.423911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.460339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.493469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 282/561] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:26.494962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.522110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7165ca-1ab5-4d58-87ab-1312a0b0598e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.558358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.559040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.559439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.559805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] PATH_INFO: `/v3/users/glance` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.568469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.569955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-5017a537-facb-4043-8de7-680c8b8a98db None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:26.582236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:26.583971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 280/562] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8caee1-9d8e-4abc-a36b-325e0869feef None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.585792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8caee1-9d8e-4abc-a36b-325e0869feef None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.591566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8caee1-9d8e-4abc-a36b-325e0869feef None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.591566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 281/563] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:26.591566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8caee1-9d8e-4abc-a36b-325e0869feef None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.620321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e29089-d1ce-4ace-b093-e630691e0f17 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.657408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.658035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.658315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.658587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.659301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:26.659607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.661001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:26.670038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 283/564] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:26.670914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.684955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.737068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8caee1-9d8e-4abc-a36b-325e0869feef None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.750562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.783836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.784488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.784768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.785055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.797583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:26.802522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:26.803633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:26.804450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 282/565] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:26.805169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.815805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc77f6f-80ec-45f1-994e-68eef2652026 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.854944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.855674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.855956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.856229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.868818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.870322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-b3985766-98d2-4caf-bf89-2eb8eecd19a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:26.886684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:26.887787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:26.888662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:26.888662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:26.888662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:26.888662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 284/566] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:26.889272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.889454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.889728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.908499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:26.909724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:26.941286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2cea2-3777-4565-a4a4-33e3ee5477bf None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:26.970688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:26.985368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 285/567] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:26.986922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:26.987704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:26.988383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:26.988674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:26.988932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:26.989710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:26.990022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:26.991598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.014475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 283/568] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:27.016714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.060349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.151008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834da2e-f6c6-4af5-834c-3dccae072068 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.154984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e938c5c-371d-4bdc-b767-4114090fbe68 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.191485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.192146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.192407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.192673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.197917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.198648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.199027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.199415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.200314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:27.200823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.202445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.204696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.206198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.212528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 284/569] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:27.213963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:27.223745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:27.225800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:27.228355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:27.228355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.228355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:27.228355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 286/570] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:27.228355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.290297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.328376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.346367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2bfd90-273b-4dd7-8f0b-fcfaed27ef65 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.362168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296841f4-fd20-48cc-a8cd-6bf58d21503a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.396888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.397999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.398583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.398885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.399631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:27.399944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.401521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.416146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 285/571] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:27.417036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.435336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.436365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.436849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.437332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.457188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:27.466659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:27.468202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:27.469444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 287/572] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:27.470270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.534514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c640a2e4-15f2-40f3-9f38-c0b1d99ce74b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.574075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.574717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.574993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.575316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.586652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.588182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.596222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:27.608081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:27.609264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:27.610354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:27.610354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.610354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:27.614051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 286/573] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:27.615100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.639793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87cdd8-6858-44d1-a22d-21c358626270 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.644219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.701009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.701996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.702362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.702637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.719926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.721475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:27.740369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:27.741410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:27.742348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:27.742348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:27.742348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:27.742348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 288/574] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:27.742584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.767977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f832c3-4b6c-415c-b5c3-f14fa0f5f268 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.820476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.821253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.821615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.822000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.822748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:27.823255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.824856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.841058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 287/575] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:27.842125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.891187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12afde12-8245-4883-acb3-80c38277a3ff None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.927916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:27.938162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:27.938868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:27.939265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:27.939638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:27.947882 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:27.949376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:27.963184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:27.965084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 289/576] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:27.966880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:27.990909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafef800-9e99-4547-a900-b8ff7c98280f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:27.994009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.044701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.045424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.045779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.046183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.047254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:28.047690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.049445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.060843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 288/577] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:28.062102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.121211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6c25a-a57d-492b-9c07-f427d5800fb8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.170311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.171286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.171736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.172158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.173223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:28.173734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.176066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.195501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 290/578] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:28.196486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.219377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32611a4-ac77-4679-9855-640d240f83a7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.250838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.271846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.272774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.273279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.273716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.274794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:28.275303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.277563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.295324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 289/579] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:28.296517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.316318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.344321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd865c8-9b53-4e01-a83c-846577b49325 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.394649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.395412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.395749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.396168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.405340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.407839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.410997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c201b6-2fc5-4fa4-8037-7ea4645a6684 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:28.427469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:28.429203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 291/580] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:28.431011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.466411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.467257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.467657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.468078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.468890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:28.469311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.471035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.485211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 290/581] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:28.486258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.537675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.561417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f661997f-8a03-4e18-9404-b72c0d6188bd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.609210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.610199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.610632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.611123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.615537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.621605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29723dfd-f245-4861-9771-e1d3db399930 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.629605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.632044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:28.652711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:28.654843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:28.655857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:28.655857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:28.655857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:28.655857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 292/582] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:28.655857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.673000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.673695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.674046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.674412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.682812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.684493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:28.697630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:28.699410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 291/583] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:28.701114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.796355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5975e68-f739-43df-9bda-20fb0bf95ccd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.842807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:28.845551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.846336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.846690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cca8edb-9ffb-4de0-882e-c1799fef29bf None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:28.846877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.847252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.857688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.860048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:28.874519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:28.876665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 293/584] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:28.878620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.895544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:28.896340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:28.896678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:28.897007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:28.897892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:28.898336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:28.900008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:28.927560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 292/585] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:28.928743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:28.949581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.011360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a4e79a-489f-4812-b3b0-6f7915852151 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.049744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.050432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.050776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.051142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.051973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=ec66fef9dc274223b34659e369063ced) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:29.052368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.052590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d0ff3-868f-444c-9a55-92a90621cb91 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.054008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.060608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 294/586] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/role_assignments?user.id=ec66fef9dc274223b34659e369063ced&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:29.061594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.107346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.108777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.109503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.110050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.111209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:29.111645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.114147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.133269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 293/587] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:29.135015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.176477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.248849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486d3f74-51f4-4b90-b233-5c29f3504fd6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.272765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.306623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.307899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.308536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.308959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.310137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:29.310646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.312852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.320222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d192d6-2a85-4f31-b634-a53ede140f48 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.341150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 295/588] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:29.341150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.378249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.382204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.382204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.382204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.383029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:29.383635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.387624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.422201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 294/589] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:29.423747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.492517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f97b268-ebf4-45b7-bea1-4d108b8c43f0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.542361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.545181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.545833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.546176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.546448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.558358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.559899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:29.579333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:29.581520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:29.583888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:29.583888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:29.583888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:29.585104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 296/590] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:29.586079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.590048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aee7-fc8f-41c2-a7af-92fd95e3b3d8 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.619862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.650359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.651400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.651774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.652170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.661004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.662681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:29.677055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:29.678896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 295/591] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:29.681241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.758141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882e0041-93fe-4bf6-9271-e74e39c7bd52 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.806680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cadd8-7b4b-45fa-a50a-83cb39d12e4d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:29.819909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.821405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.821883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.822348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/f034fd79c9ef454498ee0874d2c97d16/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.849466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:29.850291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:29.850698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:29.851111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:29.851427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=f034fd79c9ef454498ee0874d2c97d16, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.854207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.855692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:29.869377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:29.871876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:29.887122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 297/592] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:29 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/f034fd79c9ef454498ee0874d2c97d16/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:29.887122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:29.902854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:29.904915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:29.906640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:29.906640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:29.906640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:29.911687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 296/593] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:29.912955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.066228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:30.107470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998c0353-7767-48c0-98a1-cdaa70159191 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.117678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7e928-eeff-4e99-9979-36dac48a498d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.175766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.176498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.176980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.177548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.178616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:30.180258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.182924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.193002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.194458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.195297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.196122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.197632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:30.198338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.201212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.206010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 298/594] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:30.208424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.237971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 297/595] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 13:22:29 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:30.243138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.354683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:30.458458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc25a3d-5707-461c-95da-ad0cbf859fe0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.472861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b167a71e-f662-4031-83af-7f12f6acd726 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.521759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.522913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.523520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.524055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.525198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:30.525741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.528297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.528510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.529445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.529999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.530505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.531005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:30.544190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.548041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.554907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 299/596] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:30.556714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:30.571393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:30.573048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:30.574804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:30.577837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 298/597] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:30.579137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.730405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46959234-586b-44c0-8cb5-5b56bca24219 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.769010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.770296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.770906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.771453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa08545-4887-44a6-a536-e5394df41dce None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.771571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.785097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:30.794487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.796644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.822972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.823800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.824177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.824546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:30.826993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:30.828577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:30.829825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:30.829825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:30.829825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:30.832519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 300/598] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:30.833888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.844558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.846397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:30.865342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:30.866333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:30.867267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:30.867927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 299/599] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:30.869233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:30.894448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:30.956607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734ce07-d5b9-483a-af83-e5757011c71b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.988303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14c6565-5857-4979-ae1d-d3d770a4c95b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:30.995005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:30.995912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:30.996366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:30.996741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:30.997560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:30.998434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:30.999759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.019665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 301/600] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:31.020913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.030393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.031319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.031787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.032225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.033117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:31.033562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.035327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.050277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 300/601] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:31.056313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.088270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:31.136685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87568faa-cb8e-4d72-a7b2-eebc1cbc2ddd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.145264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:31.175481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed83f7e0-2a8b-4f21-9012-aa63e0175b18 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.181285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.182050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.182381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.182727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.183559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:31.183937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.185497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.198730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 302/602] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:31.200086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.223115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.224175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.224628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.225098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/656cb98cf9b147f993f2cd31ec4ea55c/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.242547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=656cb98cf9b147f993f2cd31ec4ea55c, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.244105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16578ec5-ef9b-4b70-80e3-e61afe57b04c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.262298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 301/603] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:31 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/656cb98cf9b147f993f2cd31ec4ea55c/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:31.264920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b9eb-2a3b-407f-979e-aaf1e1f4f8eb None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.265717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b9eb-2a3b-407f-979e-aaf1e1f4f8eb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.266500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c56b9eb-2a3b-407f-979e-aaf1e1f4f8eb None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.276233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 302/604] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:31.276233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56b9eb-2a3b-407f-979e-aaf1e1f4f8eb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.363709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dbcf2-82ea-4be8-b96c-fd87aaf7c085 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.430980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:31.446527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.447910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.448191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.448469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.459737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.461369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.473160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c56b9eb-2a3b-407f-979e-aaf1e1f4f8eb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-30d65546-3449-4d6e-abc1-454aab432b3a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:31.481493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:31.482564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:31.484442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 303/605] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:31.484937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.485258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.485353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.517482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:31.520580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:31.523367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.524136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.524552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.524895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.547565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=8e18ece85f014104b3c8fe92355b18a9, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.549199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.557329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:31.566934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 303/606] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:31 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:31.568118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.646052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 304/607] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:31.682775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.753171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c20322-2189-42e5-8389-3ee7fac505f2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.790107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:31.804745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.805531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.805918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.806198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.807250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:31.807622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:31.809380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:31.823848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 304/608] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:31.832210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.872138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffce7d3-5593-4f77-9797-bb9ed0c72c92 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:31.914199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:31.915272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:31.915831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:31.916160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:31.937856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:31.944605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:31.945784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:31.946913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:31.959129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 305/609] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:31.960674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:31.997583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30131db7-fdca-4f0c-adfe-e2464273879b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:32.000846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:32.060687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:32.061613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.062056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.065390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/a36d4232444f46c08f950d19ae395643/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.121262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=a36d4232444f46c08f950d19ae395643, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:32.122981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:32.143768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05c782a-4e80-4e03-97a2-41199cd39490 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:32.150805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 305/610] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:31 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/a36d4232444f46c08f950d19ae395643/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:32.196107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:32.197111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.197572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.198047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.199010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:32.199527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:32.201183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:32.212207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 306/611] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:32.218201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:32.303612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:32.419724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d629ac-5936-4e91-a38c-97e0c40c8c4b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:32.469005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:32.470115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.470623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.471153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.484121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:32.486452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-adabfba4-11a1-4a29-8942-3d61090020a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:32.515100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:32.516124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:32.517006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:32.517006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:32.517006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:32.517624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 306/612] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:32.523654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:32.623395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:32.640136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7f87d-945b-4299-bda4-15b3009dd1e2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.640616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7f87d-945b-4299-bda4-15b3009dd1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.641093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b7f87d-945b-4299-bda4-15b3009dd1e2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.644674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 307/613] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:32.650860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.651173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.651455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.661472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ef72d-f242-44a0-b162-ac205d3ef71d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:32.692417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:32.693666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:32.698613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:32.699281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.699564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.699838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:32.700495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:32.700796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:32.702200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:32.716351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 307/614] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:32.722345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:32.841688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 308/615] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:32.879337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:32.909473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3157fe-a1ff-4090-ba07-5dc855f77969 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:32.918657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:32.995225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:32.996258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:32.996610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:32.996885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.009870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:33.011413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:33.015369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:33.036545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:33.038890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:33.041003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 308/616] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:33.045346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.070601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a530dc-f7f7-4fde-8d65-969be5e16900 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:33.140419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:33.141503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:33.141912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:33.142396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.170998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:33.181366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:33.182364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:33.183229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:33.190206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 309/617] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:33.199570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.291109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df35d36-a4cd-4b05-81ed-6e1962a45628 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:33.335482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:33.343827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beace05-3db7-48ca-9969-a8112ed01fd7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:33.396034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:33.397132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:33.397420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:33.397858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.398691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:33.399213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:33.399319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:33.399926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:33.400211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:33.400499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.401071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:33.401248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:33.401524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:33.402984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:33.414024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 310/618] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:33.424357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.429818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 309/619] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:33.440203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.565047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bbb448-0583-4613-b34c-95ee9a84e139 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:33.572248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:33.625347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:33.626380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:33.636889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c85a-1597-4bef-80ff-e958d5cd12dd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:33.637806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:33.638120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.663489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:33.668175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:33.671352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:33.681187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:33.681819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:33.682055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:33.682327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:33.706186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:33.707285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:33.708169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:33.708169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:33.708169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:33.708782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:33.709956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:33.711034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 311/620] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:33.719876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:33.736953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:33.737991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:33.738889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:33.738889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:33.738889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:33.740837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 310/621] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:33.746795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:33.981613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600801d-4907-47da-8b35-8e053c86df45 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.002862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a99eb2-af21-4952-965d-d338e00479dd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.010551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.049392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.050026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.050308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.050579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.051270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:34.051574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.052991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fae189-812b-4e9d-bedb-bdafcc45034c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.063063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.063739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.064187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.064627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.065295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:34.065664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.066662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 312/622] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:34.068016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc9d9-566b-4e56-94c1-b81271a506fd None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.068384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc9d9-566b-4e56-94c1-b81271a506fd None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.068665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45dc9d9-566b-4e56-94c1-b81271a506fd None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.072538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee65c4-9a18-4304-bee2-924c5a4356e5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.074233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 313/623] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:34.075452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc9d9-566b-4e56-94c1-b81271a506fd None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.090254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 311/624] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:34.097122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.097831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.098324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.128951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:34.130504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:34.135664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.187969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45dc9d9-566b-4e56-94c1-b81271a506fd None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.224941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.225597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.225876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.226193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] PATH_INFO: `/v3/users/glance` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.234536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.235966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.240430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 312/625] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:34.241531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-bf973820-1966-456d-94a7-3ba9d269e7df None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:34.249150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:34.250150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 08 13:22:34.251133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:34.254701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 314/626] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:34.256179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f11-cbe2-4425-87c2-e882e7546f50 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.256473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f11-cbe2-4425-87c2-e882e7546f50 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.256745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04475f11-cbe2-4425-87c2-e882e7546f50 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.259284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 315/627] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:34.259996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04475f11-cbe2-4425-87c2-e882e7546f50 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.368867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.380297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256bb56b-cb00-4857-b01c-71b94dae4079 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.402301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04475f11-cbe2-4425-87c2-e882e7546f50 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.421408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.422183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.422480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.422714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.426438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.432230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.433730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a85254a6-5d39-4100-aa27-3bcaba79a286 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:34.453450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:34.454765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 313/628] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:34.455975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.457147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.457147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.467962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.469033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.469647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.470164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.471162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:34.471610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.473233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.482273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:34.483573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:34.502552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 316/629] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:34.503847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.589835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 314/630] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:34.591113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.648740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad7e35-c54f-45f8-9420-7c71d7d5a00f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.714004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.715069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.715582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.716068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.716812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.733382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d09296-b5ff-4136-a145-c571413c4b71 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.733655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:34.739346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:34.741253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:34.743045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 317/631] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:34.747947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.778621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.779718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.780279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.780785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.782153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:34.782774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:34.785511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:34.797856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:34.801170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 315/632] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:34.802393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:34.898045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0d0bf9-9dde-4767-94eb-9c1e32edea9d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.931584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1efe35-2ba7-4d2e-976a-e5a9cd5dd415 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:34.933847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.934470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.935116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.935116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.971777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:34.972503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:34.972823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:34.973219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:34.986559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:34.992618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:34.993719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 316/633] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:34.994556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.010251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.045899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.047350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8000aeb-bb6d-4b83-9e6b-4759a48e5d70 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.087096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.111233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c47246-ebdf-44c1-9a1d-a5e8f63d213c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.150822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.151478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.151751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.152020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.152688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:35.152986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.154385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.165502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 317/634] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:35.166349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.174466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 318/635] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:35.176317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca94b3-759a-42dd-8095-c66b2a052704 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.176708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca94b3-759a-42dd-8095-c66b2a052704 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.176989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca94b3-759a-42dd-8095-c66b2a052704 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.183391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 319/636] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:35.184230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca94b3-759a-42dd-8095-c66b2a052704 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.251792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.297267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84f747-1d83-4d45-9b58-7e964deda536 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.301783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca94b3-759a-42dd-8095-c66b2a052704 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.352623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.352982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.353326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.353607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.353881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/ec66fef9dc274223b34659e369063ced/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.361774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.362741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.363144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.363426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.364396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:35.364810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.366977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.377022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=ec66fef9dc274223b34659e369063ced, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.378435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 320/637] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:35.379470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.379593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.402493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 318/638] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:35 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/ec66fef9dc274223b34659e369063ced/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:35.403534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.532886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62231d66-387b-45b0-86e5-e4fe90313f44 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.554728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abe212-9017-4117-915e-ee84484a6ac6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.573882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.574636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.574999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.575314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.587942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.588177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.589846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.608667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.609453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.609739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.610012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-96d626e1-c406-4d64-a1e0-8a0f95d3c30c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:35.612637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:35.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:35.614435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:35.614435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.614435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:35.621099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 321/639] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:35.621099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.621099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.621099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.630729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.632279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.638289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:35.639957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:35.654552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:35.655635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:35.656642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:35.656642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.656642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:35.656642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 319/640] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:35.657680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.698595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.745588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 322/641] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:35.746284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.796640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a817f-2c9a-4abc-b4ab-4878a70ba0ce None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.837726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.838685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.839210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.839686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.853085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.854870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:35.871701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:35.872778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:35.873703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:35.873703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:35.873703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:35.880720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 320/642] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:35.880720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:35.918858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:35.926380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad97cd6b-625a-4b36-8f63-d921c5954ad7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:35.977555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:35.978589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:35.979094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:35.979557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:35.980613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:35.981084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:35.983409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.003889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 323/643] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:36.005499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.038863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217853e-1daa-4d34-9b06-fe3d0b0e0cf3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.065395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:36.091413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.092295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.092733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.093133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.094262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:36.094757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.097005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.120258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 321/644] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:36.128023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.197673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cbcda1-7bcf-486a-af86-f4fe4c8d77e5 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.256950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.258660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.259505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.259943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.273065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:36.279992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:36.288193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:36.290037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 324/645] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:36.292625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.310907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbb1ff-1b5f-4a93-91ee-37ed1d485f43 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.366867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.367755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.368209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.368528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.369610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:36.370110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.372269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.398679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 322/646] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:36.400017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.482110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:36.496711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da2407-5a05-4d25-b415-fcf6063c85ff None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.551346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb1532-e66f-44fb-bc9a-016786e22f20 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.561162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.561892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.562171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.562440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.572710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.574205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:36.586617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:36.587693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:22:36.588645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:36.591464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 325/647] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:36.593182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.612142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.613199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.613688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.614170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.628148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.630895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:36.653017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:36.654686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 323/648] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:36.657596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.665214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:36.728402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51471f-230e-4d77-bc11-5ad182233492 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.795487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.796459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.796865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.797353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.798533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:36.799068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.801784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.815454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211270a5-96bd-4071-b200-a6dd9cbe0181 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:36.824755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 326/649] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:36.824755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.851155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:36.885214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:36.886127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:36.886411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:36.886688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:36.901207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:36.903937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:36.925436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:36.927757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 324/650] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:36.929606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:36.970400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678ac5d-485b-468d-962d-93d1cc78d075 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.013617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.014543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.014961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.015411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.016601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:37.017041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.018842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.041676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 327/651] 10.222.0.33 () {56 vars in 1137 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:37.042785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.050815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:37.106143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5472fd-077b-456b-8639-288bae60f764 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.163014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.164317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.164800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.165275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.166584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:37.167165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.169956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.186831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:37.192977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 325/652] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:37.193998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.245549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e1e5f8-478c-431c-a62d-eeca1e8c33d6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.312091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.313302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.313891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.314460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.333452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.336540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:37.365104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:37.366452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:37.367823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:37.367823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:37.367823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:37.374011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 328/653] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:37.374212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42262a15-797d-450f-aaea-0fb7aaf05376 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.374307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.419587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:37.439563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.440798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.441410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.442017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.443675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:37.444389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.447516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.544990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5cd373-86ef-49ed-8cd5-b77e68de5dd2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.578239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:37.588996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.589631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.589909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.590180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.662478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 326/654] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:37.664141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.671242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.672636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.796314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 329/655] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:37.798138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.815288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb520b7-dc5e-4480-a49f-23a1dd48cea6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.837946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:37.862627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:37.863620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.864063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.864497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.865746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=f034fd79c9ef454498ee0874d2c97d16) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:37.866268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:37.868865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ec8e29-ef2e-46a3-8e7c-71ae116b2e94 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:37.878578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 327/656] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/role_assignments?user.id=f034fd79c9ef454498ee0874d2c97d16&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:37.880774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cd85d-7baf-4550-b821-2ae062aa499d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:37.881249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cd85d-7baf-4550-b821-2ae062aa499d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:37.881770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6cd85d-7baf-4550-b821-2ae062aa499d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:37.888810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 328/657] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:37.888810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cd85d-7baf-4550-b821-2ae062aa499d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:37.949409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8de0e-f14e-4901-af76-24a31191eeb6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:37.966070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.000358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.001558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.002020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.002431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.003340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:38.003851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.005555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.023522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 330/658] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:38.026043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.066053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6cd85d-7baf-4550-b821-2ae062aa499d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.130123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.131500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.132014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.132512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.178776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.209202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d269df-2cb6-4ca4-8c37-19e973d96d57 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.252596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.255727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.257219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.258017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.258444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.259286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.273049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.274625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-11b709d7-94e8-4c28-8256-a98562c50d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:38.291531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:38.293346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 331/659] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.295135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.315055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:38.316324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:38.353626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.427923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 332/660] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:38.429599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.460410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 329/661] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:38.462818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.565432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.596410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0aa0b-2e74-4eeb-a9ee-14c3ee1053b2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.629223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc521b-19f5-438c-a177-3ff133ba5338 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.653262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.654177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.654660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.655159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.657013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:38.657608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.661213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.674889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.676781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.677227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.677673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.683658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 330/662] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:38.685087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.698308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.701030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.702021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:38.720831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:38.722706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 333/663] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:38.724603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.852770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca5746-6b28-45f3-bd08-0907b9a5bb63 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.899457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.900493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.900899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.901299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.907739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664b6e25-c2e7-4889-b674-e7297cc25c36 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:38.919137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:38.921878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:38.942064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:38.950535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:38.952785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:38.953986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:38.953986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:38.953986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:38.955317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:38.956108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:38.956513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:38.956915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:38.959589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 331/664] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:38.960623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:38.976349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:38.985732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:38.987658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 334/665] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:38.990766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.077716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:39.155042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89839a3-c71f-4b02-b0b0-a80da8ce7a04 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.155520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3227a2-9969-4a29-b547-c0425e0761b5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.212966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.213814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.214223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.214658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.215608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:39.215791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.216122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.216717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.217199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.217673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.218117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.230669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.233454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.246267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 335/666] 10.222.0.33 () {56 vars in 1139 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:39.246267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:39.254628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:39.256431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:39.258100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:39.263546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 332/667] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:39.264780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.322278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:39.403990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485bf603-bd11-49d1-9078-79acebe650bc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.428298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e66ac-aca2-431b-a1f2-6c0dcf45c3b5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.449971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:39.474109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.475169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.475652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.476176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.477369 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:39.477907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.480790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.491220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.492458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.492928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.493386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.494619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:39.495188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.496881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.506246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 336/668] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:39.511127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 333/669] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:39.511127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.515528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.652984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fa5a5-8713-4f99-99e0-535c9af42937 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.696399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33d57bb-d362-4d66-aa4a-d180a1682f76 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.709511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:39.716301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.717908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.718239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.718556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.719928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:39.720467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.722651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.741293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 334/670] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:39.742782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:39.768725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.769742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.770114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.770566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.805236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:39.877192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.879547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.895516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b704366-0db9-4ba0-a9e2-037306c7aa4f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:39.933602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:39.934737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:39.935256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:39.935746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:39.947022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:39.949685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:39.967120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:39.968281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:39.969840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 335/671] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:39.970463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.033988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 337/672] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:40.035390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.048109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.110816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.115355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe26768-2a78-4dcc-992d-aa3ddcae62a6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.170341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.171220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.171671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.172382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.190455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.192751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.198615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d075b9-2614-4fa7-9757-b97b2a68cb68 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:40.218190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:40.219284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:40.220172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:40.220172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:40.220172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:40.220172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 336/673] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:40.220997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.260909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.261594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.262067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.262542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.263894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=656cb98cf9b147f993f2cd31ec4ea55c) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:40.264439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.267171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.280723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 338/674] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/role_assignments?user.id=656cb98cf9b147f993f2cd31ec4ea55c&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:40.284154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.394302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059aea3-1b45-445f-9d8e-ddd4d3191112 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.415121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.457728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.458708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.458999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.459335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.460309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:40.460675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.462793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.472435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51bf48-fb59-43c7-b194-62f8472948e3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.487120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 337/675] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:40.487120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.506213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.535348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.536251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.536716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.537185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.553958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.556275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-9cc11f98-1631-4187-90d0-4686a6f1415a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:40.574853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:40.575931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:40.576817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:40.576817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:40.576817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:40.576817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 339/676] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:40.577520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27887-ee9b-4283-bc9e-df6537c4a35c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.577681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27887-ee9b-4283-bc9e-df6537c4a35c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.577877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be27887-ee9b-4283-bc9e-df6537c4a35c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.583250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 340/677] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:40.583250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27887-ee9b-4283-bc9e-df6537c4a35c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.662450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062c3a64-cf45-47bf-8721-6273f5af93dd None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.706566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.707616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.708089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.708550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.709701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:40.710225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.712771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.734386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 338/678] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:40.734386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.750412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be27887-ee9b-4283-bc9e-df6537c4a35c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.782391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.803776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.804596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.804955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.805224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.806054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:40.806562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.808759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d6f7b-c3c6-468a-9adc-3fa85592c69f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.817016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 341/679] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:40.818316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.818608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.818883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.842450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:40.843762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:40.860218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:40.880384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0de83-af34-4f22-9e54-1834f0836245 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:40.930653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:40.931740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:40.932246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:40.932875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:40.934181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:40.934573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:40.936255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:40.954093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 342/680] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:40.955113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:40.960147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 339/681] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:40.965897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.088260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbaec9a-61a4-4f18-87db-38773afd071d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.114629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.136831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cf152-c479-4bfa-92a1-c73cd1caacd8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.148809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.150212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.151004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.151684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.169133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.171805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.188825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.189482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.189780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.190149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:41.196998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:41.199049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:41.201273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:41.202316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.202316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.202584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.205453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 343/682] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:41.207066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:41.217715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:41.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:41.220682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:41.224470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 340/683] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:41.226384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.389026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4b351-9d4d-43a2-b7d1-7e6adbd10e3a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.393997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ef402b-806d-4719-b260-9dd1cbc14a83 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.427806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.428803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.429265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.429674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.441685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.443486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.454389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.455675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.456265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.456790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.458159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:41.458818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:41.461602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:41.462714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:41.463812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:41.463812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:41.463812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:41.464499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.470484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 344/684] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:41.471688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.499661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 341/685] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:41.501282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.503726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.591838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.602585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a417a5-a428-49dd-a562-769f190aa299 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.642786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.643672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.644096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.644614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.645529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:41.646076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.647941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.668111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 345/686] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:41.669386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.678418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fe173-3598-4b33-878e-273d7cd47bc2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.737602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.738733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.739143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.739518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.740439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:41.740843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.743183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.762528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 342/687] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:41.763797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:41.774078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.791758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd12ddc-c334-4d24-9f6b-3d4e141c234a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.830582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.831567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.831994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.832408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.896293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db419ee-d1cf-44a3-ba4a-95dcd57b38d8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:41.910945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:41.913384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.915632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.942086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:41.942879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:41.943841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:41.944230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:41.945188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=a36d4232444f46c08f950d19ae395643) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:41.945605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:41.947555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:41.963129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 343/688] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/role_assignments?user.id=a36d4232444f46c08f950d19ae395643&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:41.963129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:42.107496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:42.140272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fdb3ee-b566-4b4e-8bd5-7fabe62a3594 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:42.146890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 346/689] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 08 13:22:42.156465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:42.223265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:42.223912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:42.224189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:42.224579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:42.225354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:42.227971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:42.269852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 344/690] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:22:41 2022] POST /identity/v3/users => generated 254 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:42.350438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe7c42-085b-45ea-bcdf-4ef0a25d64c3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:42.432742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:42.433923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:42.434536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:42.435160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:42.456964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:42.459727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1459e59f-55aa-44f5-ad95-a69cb701f29a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:42.494652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:42.495856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:42.496948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:42.496948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:42.496948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:42.503718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 347/691] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:42.509493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:42.574223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:42.701283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5bc5-1828-480c-8fc3-3634cae41f92 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:42.701527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5bc5-1828-480c-8fc3-3634cae41f92 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:42.701857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5bc5-1828-480c-8fc3-3634cae41f92 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:42.704501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 348/692] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:22:42.708995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:42.709454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:42.709609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:42.713738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4a3c5-bd8d-4eff-93aa-002f1dcba950 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:42.732521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:42.733746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:42.765739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:42.767127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:42.767403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:42.767682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:42.768503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:42.768817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:42.770552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:42.786593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 345/693] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:42.801397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:42.808316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:42.880829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 349/694] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:42.907369 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.040666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:43.067524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00b34f-a172-4f30-8f35-52aea3decf96 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.153104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f8ecd-2aca-4e21-9489-834f8119dd9e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.163944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.164748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.164990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.165415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.187976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.189940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:43.211103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:43.212357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:43.213329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 346/695] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 410 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:43.217116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.222100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.222767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.223046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.223354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.223963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.225532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:43.249384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:43.251330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:43.252953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 350/696] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 13:22:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:43.259114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.337133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:43.454941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e44ed-f7f7-45fe-8a56-c48897da059c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.469198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd6f5a9-b963-4bf1-8d7b-fe96ad147e9a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.501888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.502407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.502684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.502954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.504766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:43.505074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.506523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb1daf-8d75-4ba3-a8d4-a4d1b7ba1e97 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.519868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 351/697] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:43.521197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66143-3d8f-4602-9165-29f27d640d48 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.521483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66143-3d8f-4602-9165-29f27d640d48 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.521753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66143-3d8f-4602-9165-29f27d640d48 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.528288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 352/698] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:43.530468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66143-3d8f-4602-9165-29f27d640d48 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.536716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.537434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.537717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.537989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.538745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:43.539056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.540636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd7f55-2faa-4879-b755-d7a2fa73e757 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.554707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 347/699] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:43.560333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.560620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.560895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.581368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:43.582619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:43.586040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:43.650110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66143-3d8f-4602-9165-29f27d640d48 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.664433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 348/700] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:43.665694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.683554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:43.717320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.718457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.718586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.718849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.719559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:43.719866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.721312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.733970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 353/701] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:43.735048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.820677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a68e9-8382-46b3-8da9-f7d700c601f6 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.867027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e835a7-9fa3-4d19-a475-580757d71ca3 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:43.906278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.907196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.907658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.908078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.908983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=ec66fef9dc274223b34659e369063ced) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:43.909226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:43.909621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:43.909880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:43.910162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:43.910437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:43.911800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:43.922934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:43.927925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 349/702] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/role_assignments?user.id=ec66fef9dc274223b34659e369063ced&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:43.929201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:43.930236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:43.931184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:43.935116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 354/703] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:43.940363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:43.944947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:44.107098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa817c19-389b-42f2-808b-41133063dc23 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.147718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:44.148793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.149230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.149672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.150570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:44.151025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:44.152647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:44.163404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 350/704] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:44.172666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:44.206202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:44.304663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2611dc2-104f-4146-840f-52d504e3fc26 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.342580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:44.343650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.344271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.344764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.357300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:44.359233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-02b8a53b-c6e2-429f-b85c-ef1fa10119d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:44.392753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:44.393738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:44.394608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:44.394608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:44.394608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:44.395343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 355/705] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:44.401296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:44.467045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cafef-96b0-4b6e-bc2f-79b4aa901eed None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.467470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cafef-96b0-4b6e-bc2f-79b4aa901eed None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.467930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cafef-96b0-4b6e-bc2f-79b4aa901eed None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.470787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 356/706] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:44.476750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.477053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.477361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.492166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:44.499047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:44.500576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:44.537457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b6a7bf-ee2e-4a39-957d-e809e570fa37 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.576255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:44.577282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.577754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.578219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.579173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:44.579701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:44.581579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:44.598487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 351/707] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:44.605573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:44.685692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 357/708] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:44.719968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:44.723612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:44.736514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ddd0c9-39f9-4fc6-8d9b-36d2d110d34c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.766727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:44.786525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:44.787484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.787951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.788427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.805188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:44.806940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:44.820063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:44.821088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:44.822127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:44.824894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 352/709] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:44.838722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:44.888416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fd572-a15e-4543-9a90-af601dec7328 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.958464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:44.959184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:44.959464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:44.959785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] PATH_INFO: `/v3/services/network` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:44.960455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:44.961283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9e25b9-faea-4d04-914e-62514dc5682f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:44.961945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:44.975650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:44.976788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:44.977633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 358/710] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:44.984187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:45.006424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:45.007588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.008078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.009070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.010109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:45.010630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:45.012416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:45.034023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 353/711] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:45.039508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:45.099406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:45.174315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2844fac-9a58-4ad7-8755-43954308ab80 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:45.201461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:45.225395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:45.226544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.227052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.227565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.314746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b10c4-82b5-439f-9e10-7557480b177d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:45.315518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:45.317237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:45.371599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:45.442043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 354/712] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:22:45.447368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:45.473829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:45.474883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.477017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.477533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.478674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] RBAC: Adding query filter params (name=network) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:45.480739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:45.482403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:45.493254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 359/713] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 510 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:45.500128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:45.620301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0a298-71fc-41e9-8f72-b6062ee04c3b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:45.649057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14fb73d-99e1-41eb-b66d-81607fbb8843 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:45.674588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:45.676419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.676929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.677627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.683705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:45.690109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:45.690801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.693037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.693187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.693851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] RBAC: Adding query filter params (type=network) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:45.694221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:45.695802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9e04a-8da6-45c2-8902-366b515c278c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:45.698090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:45.700794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:45.704471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:45.707347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 360/714] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:45 2022] GET /identity/v3/services?type=network => generated 124 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:45.709083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182d94e-3a16-4bf2-a413-d30c5ca43b19 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.709673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182d94e-3a16-4bf2-a413-d30c5ca43b19 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.710016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182d94e-3a16-4bf2-a413-d30c5ca43b19 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.718033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 361/715] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:45.723863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:45.724920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:45.725861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:45.725861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:45.725861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:45.727855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:45.728200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 355/716] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:45.728310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:45.728784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:45.735493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:45.767944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:45.769757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:45.931222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 362/717] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 08 13:22:45.956953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5050b37-7566-4343-afa2-9632505f9260 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:45.960508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.004592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.005143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.005427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.005703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.006366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:46.006666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.008310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.025066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 356/718] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:46.033791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.102957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:46.160116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff77452-69bf-4d18-a4b2-4d2472ed5e78 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.207101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.207912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.208225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.208485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.217912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.220028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.222747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2634b73-332d-4069-847c-3b34ea1ed18e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:46.246744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:46.248563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:46.250401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 357/719] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:46.255146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.283593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:46.304348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.305060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.305333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.305646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.332694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:46.344096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:46.345158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:46.346436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 363/720] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:46.353679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.503120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036406dc-c5c7-48cf-b8b3-4de8fa4b153c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.537738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.539136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.539136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.539502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.540048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:46.540506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.541945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.545884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8b84c-e7f0-40f7-b4d6-70c313e33f7b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.552149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 364/721] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:46.566427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.590248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:46.631661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:46.664153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.664843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.665200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.665474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.666212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:46.666520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.668088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.695616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 358/722] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:46.702385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.850320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20143613-5c87-4faf-abb4-2459b0155e87 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.852158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9866deb9-a2ae-4726-9c3e-6db635e7e92a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:46.891527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.892357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.892734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.893094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.899021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:46.900130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.900552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.901655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.902559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:46.903545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.905486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e0434-97c4-496e-b6f1-3532bf787aa7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.905621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:46.907158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:46.913111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 359/723] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:46.914722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082484e-d39c-426f-848c-1a82ec11aedc None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.915216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082484e-d39c-426f-848c-1a82ec11aedc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.915626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082484e-d39c-426f-848c-1a82ec11aedc None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.917455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 360/724] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:46.932523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:46.933100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:46.933533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:46.942225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:46.943387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:46.944526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:46.944526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:46.944526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:46.947347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 365/725] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:46.956970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:46.964105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:46.965638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:47.011351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:47.059037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 361/726] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:47.113951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:47.184994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:47.190912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e079c47c-d483-4f5c-a8b1-fdc6e09656ed None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:47.261741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:47.263095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:47.263654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:47.264109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:47.265160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:47.265707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:47.269989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:47.303094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 366/727] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:47.309785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:47.337233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f5020-1800-425e-aae0-80a4e355c737 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:47.413235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:47.414713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:47.415435 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:47.416103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:47.417507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:47.420303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80a4367-8898-4154-8420-1cd33c33849e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:47.456637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 362/728] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:22:47 2022] POST /identity/v3/services => generated 251 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:22:47.460922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7acd19-f089-491a-a35c-a89294e81103 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:47.461952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7acd19-f089-491a-a35c-a89294e81103 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:47.463187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7acd19-f089-491a-a35c-a89294e81103 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:47.467807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 363/729] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:47 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:47.477286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:47.478569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:47.479401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:47.517125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:47.520000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:47.575752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f773b08-83b7-45b8-8074-f1e629b07a64 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:47.589521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:47.698748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:47.700760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:47.701373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:47.701494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 364/730] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:47.701942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:47.711861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:47.713459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:47.725886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:47.727239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:22:47.728305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:47.731492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 367/731] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:22:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 423 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:47.733297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:47.746670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:47.788564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:47.946512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655189b-f9fd-4f8f-b601-715bba664039 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:48.030852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a513810-6578-42ea-b098-b6f1e487a050 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:48.067564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:48.069192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.070103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.070896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.072687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:48.073658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:48.076800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:48.101172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:48.102259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.102748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.103275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.123057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 365/732] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:22:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 390 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:48.131609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:48.143000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:48.145422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:48.147349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:48.149153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 368/733] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 402 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:48.162416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:48.196550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:48.415455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:48.482807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bed9524-bf0c-48af-a048-5b1b672f8e0c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:48.498107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a14f6c-3ee7-4f43-864d-71ad785ee38e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:48.544010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:48.544327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.544474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.544824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.545416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:48.545711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:48.547168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879278f0-3515-4f64-9b8b-eafea9ba23ef None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:48.556518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:48.563114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 369/734] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:48.563114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f0df-f114-4011-8329-9c33831056da None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.563114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f0df-f114-4011-8329-9c33831056da None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.563114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2f0df-f114-4011-8329-9c33831056da None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.564931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.565159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 370/735] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:48.566034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.566491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.566774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.567038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.567705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.610409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:48.614882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:48.694631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:48.697201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 371/736] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:48 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 08 13:22:48.698072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:48.699507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:48.733875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:48.835277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:48.874721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ddf66-7764-486f-ab71-cb935e0116a9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:48.878807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 366/737] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:48 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 08 13:22:48.880800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:48.928815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:48.929904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:48.930403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:48.930893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:48.950354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:48.953153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:48.980580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:48.981706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:48.982603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:48.982603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:48.982603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:48.992651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 372/738] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:48.994749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.030389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfc44d2-9fe3-41d9-9fa9-caa139d4d83c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.050242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:49.076293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.077929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.078437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.078921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.079832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.081921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.102568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 367/739] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:22:48 2022] POST /identity/v3/services => generated 240 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:49.103895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.236215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95875b-652a-477e-932a-bd39e4171341 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.250856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fde227-2f72-4d37-868e-efb52bf44802 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.284963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.286024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.286513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.287489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.288460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:49.289035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.290792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.312143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 368/740] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:49.318654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.333307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:49.356159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:49.405204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.406856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.407654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.408471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.446563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ed97-49d6-451c-b928-6ce9bfaa6e06 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.449215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.453075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.489394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:49.489925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:49.491846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:49.493684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:49.493684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:49.493684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:49.494027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.494027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.494027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.498269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 373/741] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 506 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:49.504299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.511815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.513406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:49.528750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:49.530197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:49.531627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 369/742] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:49.535907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.649716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:49.662361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145b2d7-f68b-44dd-8204-fa449deac9d4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.686128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ab838-6d04-421c-9f57-a22db1344bbc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.702233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.704738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.705248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.705700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.706731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:49.707243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.708905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.732197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 370/743] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:49.745694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.758057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:49.759961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.760498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.760990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.761893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:49.762371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:49.765059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9083fc-054b-4f19-97f4-c69f1b9095b5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:49.788001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 374/744] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:49.791255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c602-3119-4bb9-a9a2-d029eee56e7f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:49.792028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c602-3119-4bb9-a9a2-d029eee56e7f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:49.792620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c602-3119-4bb9-a9a2-d029eee56e7f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:49.794970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 375/745] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:49.801088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c602-3119-4bb9-a9a2-d029eee56e7f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:49.853572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:49.961487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a06811-e39a-49c1-93ef-72fa8c5151df None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:49.989891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c602-3119-4bb9-a9a2-d029eee56e7f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.004177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.005310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.005811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.006299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.028235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.056045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.056717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.057010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.057267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.065803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.067550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-999b794d-0523-4f00-a8c8-ac220b173d77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:50.080793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:50.081823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:50.082725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:50.087116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 376/746] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:50.088608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.088893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.089165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.100328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.102284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.110139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:50.111571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:50.193579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 377/747] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:50.194595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.223445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.225704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 371/748] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:22:50.233019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.288573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.313277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52004fd1-456b-412d-8d75-e771fc0e8236 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.347503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3928a2f-c725-4701-af63-d57f682bfc22 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.351692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.352389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.352670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.352937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.353702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:50.354009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.355540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.369693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 378/749] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:50.375619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.402993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.404092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.404665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.405146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.417364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.419436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:50.436361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:50.437416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:50.438299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:50.438299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:50.438299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:50.445069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 372/750] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:50.446543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.476610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.511483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bc08c-88d0-4b15-a5e9-d58f078e2c36 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.572361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.572923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.573201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.573742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.574451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.578941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b4b5fd-f0cc-4b3a-ad1d-204bd4eddaa7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.587019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:50.592288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:50.593668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 379/751] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:50.595106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.634010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.634635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.634909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.635256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.636001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=6afd5333a03e4e098460fbbeef927d1b) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:50.636351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.637868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f89447-f192-4c74-ab42-2628c7925f1f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.651116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 373/752] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/role_assignments?user.id=6afd5333a03e4e098460fbbeef927d1b&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:50.651116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba145-ea66-4fea-9835-4a8774d30782 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.651116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba145-ea66-4fea-9835-4a8774d30782 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.651116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ba145-ea66-4fea-9835-4a8774d30782 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.651116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 374/753] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:50.652571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ba145-ea66-4fea-9835-4a8774d30782 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.744327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42c2875-55a5-476f-a8d2-3dc96d145c57 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.803588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.804376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.805190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.805479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.806392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:50.806883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.809035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae26eb-f5be-4d6c-a385-f945adfc9da5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.811395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ba145-ea66-4fea-9835-4a8774d30782 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:50.832144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 380/754] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:50.834547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372c00-201a-43cd-8864-80239276893d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.835055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372c00-201a-43cd-8864-80239276893d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.835963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04372c00-201a-43cd-8864-80239276893d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.847423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:50.874043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-04372c00-201a-43cd-8864-80239276893d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:50.876384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-04372c00-201a-43cd-8864-80239276893d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:50.884038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:50.885084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:50.885580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:50.886043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:50.887344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:50.887884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:50.892525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:50.911971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 375/755] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:50.913939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.975472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 381/756] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:50.976711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372c00-201a-43cd-8864-80239276893d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:50.989347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.103833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae518f8-848c-4b98-a7ca-b1084b9a302d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.118912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04372c00-201a-43cd-8864-80239276893d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.147182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.148533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.148966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.149385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.159008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.161506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.164601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.172950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.173635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.173935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.174210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:51.177582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:51.178596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:51.179713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:51.182321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 376/757] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:51.183522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.191383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.193134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:51.210056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:51.211389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:51.212337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:51.212337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:51.212337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:51.217726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 382/758] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:51.218774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.304477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.335251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fde504-6fd1-4670-9c24-db98dcc152b2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.365505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756d714-e310-48a0-b102-8078455a1113 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.402671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.403628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.404054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.405209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.406000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.407976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:51.423956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:51.425207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:51.426233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:51.428953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 377/759] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:51.430033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.436466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721aa0-b422-4283-8600-15901f770134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.437247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721aa0-b422-4283-8600-15901f770134 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.437617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721aa0-b422-4283-8600-15901f770134 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.437986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98721aa0-b422-4283-8600-15901f770134 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.438796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721aa0-b422-4283-8600-15901f770134 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:51.439232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721aa0-b422-4283-8600-15901f770134 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.440859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98721aa0-b422-4283-8600-15901f770134 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.457300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 383/760] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:51.458796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721aa0-b422-4283-8600-15901f770134 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.516119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.587548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98721aa0-b422-4283-8600-15901f770134 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.600383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519815fa-1ebb-4567-9e4b-0ba3889b257d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.643235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.644120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.644546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.644675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.645100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.645497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.646043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.646212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:51.646491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.646813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.647657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:51.648166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.648902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.650126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.656143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 384/761] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:51.657341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.673179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 378/762] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:51.674166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.688644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.800737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ce61f-cb25-4802-9e35-58930edf9e45 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.811573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ad14e-92f0-4236-be6f-b3cf360bd5d8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:51.816460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:51.848097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.848721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.848996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.849524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.850154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=ec66fef9dc274223b34659e369063ced) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:51.850430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.851985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.854877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:51.855862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.855862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.856137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.856683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:51.856985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:51.858396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c22962-52f1-410c-8568-73f6d28499bb None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:51.858931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 385/763] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/role_assignments?user.id=ec66fef9dc274223b34659e369063ced&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:51.859957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:51.870796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 379/764] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:51.872559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce96f-95ff-4770-8dfe-c6d970f3e04a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:51.873082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce96f-95ff-4770-8dfe-c6d970f3e04a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:51.873528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddce96f-95ff-4770-8dfe-c6d970f3e04a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:51.875488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 380/765] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:51.878977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce96f-95ff-4770-8dfe-c6d970f3e04a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.014810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f87a8d-f2ce-4806-b092-3792b7bbde8b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.039834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:52.060579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.061426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.061734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.062506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.073653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.077629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.097626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddce96f-95ff-4770-8dfe-c6d970f3e04a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-46fa8a9b-d000-45ab-9559-ad098a8f40d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:52.098557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:52.099539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:52.100477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:52.107375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 386/766] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:52.107375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.107375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.107375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.139152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:52.140755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:52.160824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.161754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.162188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.162688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.163772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] RBAC: Adding query filter params (interface=public, service_id=acf5b0715d78492f9e1052c43ecb0e4b, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:52.164374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.166106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.182674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 381/767] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:22:51 2022] GET /identity/v3/endpoints?service_id=acf5b0715d78492f9e1052c43ecb0e4b&interface=public®ion_id=RegionOne => generated 194 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:52.182875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:52.184081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.243989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 387/768] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:52.245912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.344312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded6afb9-56e8-4f4b-b9d8-6d929429d9a5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.423493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.424705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.425176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.425695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.427557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:52.427557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.430609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.446558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:52.473205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 382/769] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:52.473205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.490075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313fee28-a78e-4d1f-98bd-abf4fc9166aa None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.556031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.558138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.558891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.559693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.561131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.563758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.574307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 388/770] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/services => generated 772 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:52.576185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.600645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ef054-0c89-47ef-a68b-068e262e3a07 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.633782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:52.641936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.642652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.642934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.643251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] PATH_INFO: `/v3/services/network` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.643857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.645328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:52.655438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:52.657339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:52.658898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 383/771] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:52.663033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.734162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:52.793528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573f10-6b93-4f7c-af8e-3962da61d8e6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.865606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f025474-b0dd-4256-92f6-7bcd60ff6a12 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:52.873239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.874794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.875589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.876341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.898259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.900327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-dc5ac765-4798-4fa4-83bd-d3c19d6318e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:52.917719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:52.918727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:52.919648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:52.919648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:52.919648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:52.921362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 389/772] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:52.923382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ddba4-d5b3-4110-974b-71e5ad2e1737 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.923882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ddba4-d5b3-4110-974b-71e5ad2e1737 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.925211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ddba4-d5b3-4110-974b-71e5ad2e1737 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.926590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 390/773] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:52.928321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ddba4-d5b3-4110-974b-71e5ad2e1737 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:52.939118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:52.939832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.939938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.940469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.940857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] RBAC: Adding query filter params (name=network) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:52.941332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:52.942825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ca594a-69b0-4be4-b915-fc1e5b7ee608 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:52.950662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 384/774] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:52.950662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:52.950912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:52.951686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:52.979184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:52.981424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:53.083013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 385/775] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:53.085124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.108663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.154424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ddba4-d5b3-4110-974b-71e5ad2e1737 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.161185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.208921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.209534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bd696-f089-43c1-9351-6554b6a88039 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.210581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.211487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.212326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.213463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:53.214077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.215967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.232333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 391/776] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:53.233736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.252296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.253240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.253707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.254194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.255465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:53.255985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.259063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.272982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 386/777] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:53.272982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.375428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2042e91-62a9-4149-8ffa-fcc2d53db7c5 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.419635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.420314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.420612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.420883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.429112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:53.438989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:53.440518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:22:53.442365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:53.450909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 392/778] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:53.450909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.466182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4aa57b-0290-43fc-be7b-1470999a40c4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.497686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.522944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.523734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.524013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.524276 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.531903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.533545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:53.546185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:53.547246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:53.548149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:53.552312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 387/779] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:53.553316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.603306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e07a69-1b75-41bd-9b53-bcd9534cfa5c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.670173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.672053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.672604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.673078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.675132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] RBAC: Adding query filter params (interface=public, service_id=268614d2901d4d7d9c788902faeaf082, region_id=RegionOne) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:53.675949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.678461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.681829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684addee-5324-4719-89ee-ad3756f6c79c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.688666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 393/780] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/endpoints?service_id=268614d2901d4d7d9c788902faeaf082&interface=public®ion_id=RegionOne => generated 194 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:53.689652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.744352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.745821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.746317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.746789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.748292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:53.748762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.751432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.770749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 388/781] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:53.772682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.777844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.848203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:53.856744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696458b-eb89-438e-b6b1-5e8dea73a198 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.898528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.898528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.898924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.898924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.899693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:53.899991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.901440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:53.921109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 394/782] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:53.922061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:53.945577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e82cd79-2b3f-4534-ab22-3fa67e36405e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:53.992061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:53.992977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:53.993409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:53.994205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:53.995034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:53.997299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:54.005596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 389/783] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:54.007210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.111970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9051ca4-6c89-4dac-bd84-fc1439c4532e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.164714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:54.196089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:54.196939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.197435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.197953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.219876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:54.223160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:54.238629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a485d915-ffdf-45b8-8d7a-99a0727d44d0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.242942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-add63fd5-2728-4d09-9c93-cbdc0aa0c943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:54.253137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:54.255924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 395/784] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4b4bd-3ad9-4ddc-8e01-1d8bcc658b59 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4b4bd-3ad9-4ddc-8e01-1d8bcc658b59 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.258857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf4b4bd-3ad9-4ddc-8e01-1d8bcc658b59 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.261560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 396/785] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:54.268395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4b4bd-3ad9-4ddc-8e01-1d8bcc658b59 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.294093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:54.294775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.295187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.295360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.297500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=8e18ece85f014104b3c8fe92355b18a9, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:54.298984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cbf0e1-a7cb-4477-aa94-632d9df68d8b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:54.314517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 390/786] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:54 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:54.316160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.316452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.316725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.338538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:54.339935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:54.416179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf4b4bd-3ad9-4ddc-8e01-1d8bcc658b59 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.427116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 391/787] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:54.454927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.461877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:54.462843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.463329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.463786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.464885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:54.465379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:54.467867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:54.490023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 397/788] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:54.499774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.594991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:54.656664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:54.671117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aee4a6-e918-4823-a048-6ae8b334e70e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.713879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd25295-5f85-4022-8d45-23f5cf630e2c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.760200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:54.761296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.761804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.762284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.771414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:54.771903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:54.772656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:54.772941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:54.773216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:54.774355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:54.800921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:54.801936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:22:54.802870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:54.805379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:54.808768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 398/789] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:22:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:54.812804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:54.813818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:54.814806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:54.817163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.822355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 392/790] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:54.841539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:54.961620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e09d5f4-4093-47d4-a153-79fea388f7dc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:54.975652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:55.010771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.011445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.011725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.011992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.012784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:55.013090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.014623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.054459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 399/791] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:22:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:55.064450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.089147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f2278-e9f5-4750-b755-6dba12e0d6b8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.160922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.162679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.163825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.164709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.166248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:55.167243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.170362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97721ae-8f0e-48db-ad35-62836d6e39bc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.188829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 393/792] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:55.191907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa610-8580-4770-b99b-f3a83ff936de None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.192942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa610-8580-4770-b99b-f3a83ff936de None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.193891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7aa610-8580-4770-b99b-f3a83ff936de None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.203232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 394/793] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:55.205551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aa610-8580-4770-b99b-f3a83ff936de None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.216624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8adb433-718c-4b07-a495-1348397e64c6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.244674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:55.263385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.264038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.264323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.264589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.276572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.278156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7c4ef333-d35e-4aed-b3ba-00e6f0d2c046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:55.294549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:55.296566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 400/794] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.298375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.312317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:55.325833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:55.327502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:55.404956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 401/795] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:55.405935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.414409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7aa610-8580-4770-b99b-f3a83ff936de None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.453292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.454075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.454349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.454619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.466637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.468137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:55.484846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:55.486671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:55.490128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:55.490128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:55.490128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:55.490128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 395/796] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:55.494918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.523610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd986701-2ea5-41e8-b519-c54f99e8b387 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.572050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.572982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.573276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.573651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.574717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:55.575021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.576534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.578848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:55.607278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 402/797] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:55.609306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.647107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:55.705779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f430d-3a91-41a7-aac0-540f7bfb4988 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.764735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.765638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.766070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.766524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.767573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.770045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:55.786313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:55.788361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 08 13:22:55.789913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:55.795472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 396/798] 10.222.0.33 () {56 vars in 1129 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:55.797055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.809013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c85247-4bce-4d8c-98d9-1759fa37edc9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:55.874599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:55.876197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:55.876916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:55.877667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:55.879036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:55.879961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:55.882872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:55.906169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 403/799] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:55.908360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:55.974766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446bd81-8a34-4385-9d1c-4460d7ce8203 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.010840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.027420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.035045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.036111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.037024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.037991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.050900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.053546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:56.074274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:56.076039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 397/800] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:56.077744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.111132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95049b3-c4c0-4c94-b5f1-b9538cac0687 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.180106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.180946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.181645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.183382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:56.184861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.188456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.199567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 404/801] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:56.202067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.290631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dbd38-5707-4336-be87-48e2b350896b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.379194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.382227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.382716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.383570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.400911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.403619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:56.430178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:56.431927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:22:56.433451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:56.438040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 398/802] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:56 2022] GET /identity/v3/users/swift => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:56.440116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.449114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b312046-7486-42e5-a3fa-b9fc9f998e9b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.473102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.500295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.501306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.501756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.502207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.503501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:56.504030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.506570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.532701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 405/803] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:56.532701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.546085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.630502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624a45f-2a7b-4332-93f5-dacca2c6478e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.683405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.684486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.684998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.685443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.686620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:56.687151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.689336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2e90-7036-4c5e-b1e1-5c1e1b862694 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.703128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 399/804] 10.222.0.33 () {56 vars in 1124 bytes} [Mon Aug 8 13:22:56 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:56.703128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f9f5-5efc-460e-b604-38fef563706f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.703128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f9f5-5efc-460e-b604-38fef563706f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.703128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac82f9f5-5efc-460e-b604-38fef563706f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.703128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 400/805] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:56.703946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f9f5-5efc-460e-b604-38fef563706f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.723841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825efdc-4ddc-49da-943d-fd74e864fb88 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.787319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.788255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.788705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.789126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.790314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:56.790803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.793326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.833738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 406/806] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:22:56 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:56.833738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:56.884998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.888052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac82f9f5-5efc-460e-b604-38fef563706f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:56.919292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:56.941869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:56.942608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.942886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.943195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/6afd5333a03e4e098460fbbeef927d1b/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:56.961802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=6afd5333a03e4e098460fbbeef927d1b, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:56.963340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd101e-32a0-4b56-a4bc-7fe0a076dfd2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:56.995202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 401/807] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:56 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/6afd5333a03e4e098460fbbeef927d1b/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:56.997576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:56.997959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:56.998423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.021914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a9bddc-a1f1-40ec-9307-b5d43f33f8ac None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:57.033052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:57.034983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:57.084497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:57.085511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.086024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.086501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.087704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:57.089932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:57.135713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 407/808] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:22:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:57.137248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:57.230570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 402/809] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:56 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:57.293124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:57.365461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:57.414272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6064d-1ed5-4a12-a1e6-f28d5a9a93a1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:57.492974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:57.494031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.494496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.494962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.516196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:57.519126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:57.546811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:57.548619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:57.551989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:57.551989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:57.551989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:57.552624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:57.552843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 408/810] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:57.560023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:57.566588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e697ee6-d2aa-4d66-b397-797b8938eba4 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:57.645610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:57.646510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.646976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.647535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] PATH_INFO: `/v3/services/network` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.648523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:57.651250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f76afdab-d2d1-4fb3-83bd-74805daacbad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:57.667864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:57.671503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:57.677466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 403/811] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:57 2022] GET /identity/v3/services/network => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:57.688049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9eaa-93c1-46d2-8a0c-6509b09863ef None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.688049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9eaa-93c1-46d2-8a0c-6509b09863ef None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.688049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce9eaa-93c1-46d2-8a0c-6509b09863ef None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.688049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 404/812] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:57.688049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce9eaa-93c1-46d2-8a0c-6509b09863ef None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:57.749200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8df4-867d-4a28-92dc-2d3b44854ff2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:57.814298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:57.815494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.815941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.816434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.817536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:57.818016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:57.820696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c7e218-a12b-4ef2-9926-f950f2f92b2b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:57.850493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 409/813] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:22:57.852837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.853277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.853708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.867583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:57.879226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce9eaa-93c1-46d2-8a0c-6509b09863ef None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:57.891055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:57.893343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:57.941355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:57.942448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:57.942913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:57.943477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:57.944692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] RBAC: Adding query filter params (name=network) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:57.945182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:57.946993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:57.959298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 405/814] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:57.961066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:57.991981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.039518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 410/815] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:57 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:58.040561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.175682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910463e6-2f2e-404a-bb82-05d8e3a09f97 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.182363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92cd9ff-0c9a-4605-ba75-9cf189a291bb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.221848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.222375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.222656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.222932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.223627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] RBAC: Adding query filter params (type=network) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:58.223927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.225703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.227117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.231609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 411/816] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:22:58 2022] GET /identity/v3/services?type=network => generated 350 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:58.237049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.255064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.256870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.257606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.258838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.273464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.276998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:22:58.300128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:58.302415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:22:58.304678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:58.315509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 406/817] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:22:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:58.317515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.346899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda91a11-3f9e-4a9e-913b-c2169f1ac281 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.392776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.393661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.394081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.394490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.418686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:22:58.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:58.434350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:22:58.435221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:22:58.439271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 412/818] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:22:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:58.440661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.463836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.507495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4cf93-b289-49b7-bb82-382ae8152ad8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.541046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.547213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.548236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.548717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.549217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.550093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.551828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.587357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 407/819] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:22:58 2022] POST /identity/v3/endpoints => generated 337 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:22:58.587357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.612185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae7df8-5900-437f-822a-60b9c4f235fc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.703834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.704474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.704748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.705032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.705759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:58.706062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.712029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.735403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 413/820] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:22:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:58.746089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.817832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077ebbbb-2dca-4cdb-a859-61be762e04ce None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.865815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.866802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.867037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.867339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.868045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:58.868380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.870212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.873929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.891153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 408/821] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:22:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:58.892188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:58.914014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5802105-00b4-4c9a-8728-065a7139ad88 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:58.930390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:58.958072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:58.958759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:58.959041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:58.959342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/ec66fef9dc274223b34659e369063ced/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:58.961553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=ec66fef9dc274223b34659e369063ced, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:58.963163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:58.981106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 414/822] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:22:58 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/ec66fef9dc274223b34659e369063ced/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:22:59.035049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76cd1a-e140-43d2-b67a-f8f5f69c229c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:59.080390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:59.081552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.082203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.082710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.099199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:59.101326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:22:59.121520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:59.122534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:22:59.123523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:22:59.123523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:22:59.123523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:59.125877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 409/823] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:22:59.132506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:59.255736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f4dfe-4ff9-4467-a91c-760659a64588 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:59.296737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:59.297365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.297689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.297959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.298634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:59.298914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:59.300370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa251a1d-a3e8-4c77-988d-63ed9ce03ca4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:59.318853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 415/824] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:59.324584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:59.410883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:59.503458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfcd42-1d7c-4ed0-8022-2995645ebb77 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:59.564862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:59.565808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.566279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.566597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.576202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:59.578955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:59.587579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a83f-b3e2-47fa-9c98-ec9f87319e4b None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.588042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a83f-b3e2-47fa-9c98-ec9f87319e4b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.588523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a83f-b3e2-47fa-9c98-ec9f87319e4b None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.591379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 416/825] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:22:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:22:59.598016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:22:59.599029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:22:59.600061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:22:59.602554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 410/826] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:22:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:22:59.606585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.606585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.606585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.607963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:59.641100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:22:59.643939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:22:59.701776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:59.783869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 417/827] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:22:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:22:59.794676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d16f45-1f3e-4fa7-ba48-d125cef21575 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:22:59.822029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:22:59.822591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:59.845691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:22:59.847278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:22:59.847958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:22:59.848597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:22:59.850235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:22:59.851018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:22:59.853905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:22:59.889475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 411/828] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:22:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:22:59.898246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:22:59.956298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fa07eb-e339-450a-b5fc-d16b5d78e421 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.002329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.003187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.003475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.003743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.018445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:00.025031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:00.026928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:00.028669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 418/829] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:22:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:00.039752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.082719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c91adf-eace-43b6-a11e-2e52d4052f90 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.089954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:00.131378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.132479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.132822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.133640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.178923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:00.220287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:00.221892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:00.241710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235e96b-9991-478d-bdf4-34ce517b656d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.310021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.311507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.312291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.313035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.314580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:00.315520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:00.318446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:00.342846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 419/830] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:00.344946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.347552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 412/831] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:22:59 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:00.356683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.491388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:00.515698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:00.541707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c297479-8a3c-42c8-a6f3-7a1d3938a1ec None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.542786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ecf1e0-8f72-4056-9519-32d4a7a93d45 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.583158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.583808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.584183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.584427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.597984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:00.599518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:00.604747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.605829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.606297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.606769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-85dcda3c-c062-464a-b1c5-890c7548154e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:00.616613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:00.618522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 420/832] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3e44-b67c-4ff5-ae56-1150d6645831 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3e44-b67c-4ff5-ae56-1150d6645831 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.620340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d3e44-b67c-4ff5-ae56-1150d6645831 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.628199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 421/833] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:00.628889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d3e44-b67c-4ff5-ae56-1150d6645831 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.636601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:00.639401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c0bc6b6a-716c-4b98-9d4d-861d18acf09d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:00.666797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:00.668532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:00.671289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:00.671289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:00.671289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:00.671289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 413/834] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:00.671721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.672016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.672374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.703182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:00.705270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:00.776540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d3e44-b67c-4ff5-ae56-1150d6645831 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:00.843115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 414/835] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:00.843115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.852568 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:00.853622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:00.854104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:00.854629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:00.855741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:00.856472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:00.858727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:00.874396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:00.883541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 422/836] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:00.884747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:00.947925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.018229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cd27c4-6753-43d1-9750-dca177373ea7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.049538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df113802-5180-492c-82e8-5da72ffee9e7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.067621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.068337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.068788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.069424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.070570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:01.071000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.073189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.094910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 415/837] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:01.096430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.118907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.119928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.120331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.120790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.121810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.123986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:01.138936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:01.140941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:01.143167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:01.147931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 423/838] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:01.149151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.233696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.267499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e4f667-6381-49e9-b2c9-a26d61adbc2c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.279864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.297300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96f64d-421a-4a22-8d32-fa51deb62dbc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.313924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.314670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.315028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.315476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.324454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.326116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:01.339915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:01.342035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 416/839] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:01.344601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.346748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.347507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.347846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.348176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.356335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.357788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-bbad9dbd-b2f4-489e-82fd-c47de78a02db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:01.374086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:01.375667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 424/840] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794749e5-ba41-43b6-9580-3d6ddd6f7c73 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.377360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794749e5-ba41-43b6-9580-3d6ddd6f7c73 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.378510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794749e5-ba41-43b6-9580-3d6ddd6f7c73 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.379483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 425/841] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:01.380347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794749e5-ba41-43b6-9580-3d6ddd6f7c73 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.501149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394b33e-6321-443e-800d-12862bc1f6e3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.540772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794749e5-ba41-43b6-9580-3d6ddd6f7c73 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.548768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.549791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.550232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.550664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.551823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:01.552316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.554559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.569253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 417/842] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:01.569253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.578115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.597554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.598530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.598970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.599451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.600580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:01.601063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.603556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517977da-23b1-4d22-ab7a-b6d0bb909a16 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.620617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.634165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 426/843] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:01.636253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.636601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.636899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.657589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:01.659034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:01.717235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b96ed9-d141-422f-b1ed-41237b13cd66 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.746968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 427/844] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:01 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:01.748076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.769658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.770683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.771196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.771662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.773346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:01.773798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.776013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.790561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 418/845] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:01.791477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.885480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2962cc-f92b-4dc8-8217-243f52e9a1a3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.897114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.918287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:01.926983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.927661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.927944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.928217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:01.928942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:01.929254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:01.930662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:01.930802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83187c9-f2cb-487c-9b7b-d6c72a482008 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:01.935935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 428/846] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:01.936804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:01.995842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:01.996915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:01.997410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:01.997885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.083460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e14f89-2a78-4b6b-9fc1-da21bb14bf80 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:02.104236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:02.106192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:02.150832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:02.152090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:02.153643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:02.153643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.170204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:02.176438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:02.178248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 429/847] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:02.180009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:02.223964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:02.274109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:02.289712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 419/848] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:01 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:02.289712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:02.346972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125875a3-3526-46a7-8449-9a945dac3799 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:02.418845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:02.419775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:02.420256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:02.420746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.422415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:02.422689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:02.425467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:02.447190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 430/849] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:02.447190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:02.468007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5eacda-a83c-4c96-9e39-e7f58d297f42 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:02.534799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:02.536985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:02.537665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:02.538357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.539736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:02.540469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:02.543041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:02.563557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 420/850] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:02.571253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:02.659822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:02.676111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:02.681263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431cd59-32cf-4ae8-a385-d1618ec89287 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:02.733476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:02.734144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:02.734296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dbc5d1-6c38-423c-8a17-631bfc34f983 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:02.734439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:02.734749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.763257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:02.764846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:02.793856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:02.795723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:02.797431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:02.797431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:02.797431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:02.798623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 431/851] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:02.807370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:02.810563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:02.811677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:02.812133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:02.812596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:02.840935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:02.844869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:02.877224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:02.879065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:02.881324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:02.881324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:02.881324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:02.885065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 421/852] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:02.895370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.016174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8f8f8-736a-411c-9f61-42ec4b0dc6ec None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.048826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9571e-adde-4038-912b-dca07a89992a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.062112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.078981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.080550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.081278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.082076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.083497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:03.085225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.087969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.097431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.098062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.098342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.098620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.099335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:03.099640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.101286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.114070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 432/853] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:03.117702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 422/854] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:03.120046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.123827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.148969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.304363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3216b0-fbf9-4d02-842f-eae8c413848b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.306813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6635d5e-42e9-4912-9a8c-7eaedb00d260 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.369881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.371316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.372203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.372632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.373016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.373233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.373477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.374053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.387896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.390922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.396587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.399849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:03.414134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:03.416102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:03.417943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:03.420564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 433/855] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:03.428455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:03.430203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 08 13:23:03.432035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:03.433033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.436275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 423/856] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:03.443224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.478986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.533897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.570323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901ba67-a9f4-4146-9bf7-8b97bdc23693 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.619884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.620865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.621352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.621458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741c4e5-ac2a-444c-ace9-c2b9483843b1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.622259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.623228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:03.623714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.625577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.642095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 434/857] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:03.651220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.693807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.694989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.695855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.696325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.697758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:03.698459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.701411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.736546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 424/858] 10.222.0.33 () {56 vars in 1135 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:03.743298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:03.801466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cfcd96-23d3-496e-8505-35fad66f3aaa None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.834855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.841508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.842628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.843164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.843736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:03.844724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=6afd5333a03e4e098460fbbeef927d1b) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:03.845241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:03.846969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:03.854275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 435/859] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/role_assignments?user.id=6afd5333a03e4e098460fbbeef927d1b&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:03.894707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:03.915268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b6538-a1c1-4bc5-8af6-51cdc19b73fc None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:03.957185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:03.957833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:03.958108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:03.958379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:04.102008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:04.105268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245cb809-1832-4529-afaf-25d01dc80338 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:04.278561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:04.311925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 425/860] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:03 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:04.321898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:04.501751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def11e58-a43d-4dc8-a541-6a175eb417f6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:04.566959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:04.568595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:04.569574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:04.570349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:04.580341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c685969-df53-4612-8338-7902632e3a2f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:04.580609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c685969-df53-4612-8338-7902632e3a2f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:04.580972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c685969-df53-4612-8338-7902632e3a2f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:04.582743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 426/861] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:04.589094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:04.589094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:04.589502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:04.599432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:04.602957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:04.610521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:04.611833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:04.636799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:04.638597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:04.640967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:04.640967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:04.640967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:04.643010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 436/862] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:04.644074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:04.702509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:04.771264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b8097-5042-4358-aee9-b3ee883fb657 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:04.781193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 427/863] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:04 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:04.805014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:04.829134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:04.830159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:04.830658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:04.831180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:04.832078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:04.832618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:04.834245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:04.854481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 437/864] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:04.855881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:05.023449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1886f-d75d-417f-9d76-d917154d66cc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.059677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:05.075551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d83ee-9210-4ba8-b227-ee777772ec5d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.095063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:05.099021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.100792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.101649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.102609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.105124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.108683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c227bdd-d9fc-4783-973a-2012a66093c2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.145959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 428/865] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:04 2022] POST /identity/v3/services => generated 251 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:05.163925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.165037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.165549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.166052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.180142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.183102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:05.206512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:05.208336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:05.210201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 438/866] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:05.214170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1b88-a97c-4b51-93ac-cd900050698f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.214569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1b88-a97c-4b51-93ac-cd900050698f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.215119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061d1b88-a97c-4b51-93ac-cd900050698f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.217743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 429/867] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:05.223436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:05.226999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.227530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.228016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.268656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:05.271159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:05.420332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4ea1-d9b8-4b73-ac96-2c0bfc46b7d7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.432928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 430/868] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:05 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:05.467319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.467967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.468250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.468524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.469265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:05.469630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.471276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.473490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:05.491203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 439/869] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:05.501020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:05.546739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:05.640762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3465da3-8605-453f-81d6-c56fd026877f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.676416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83e51db-3285-4f47-b0c3-f8b3414175aa None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.683979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.684623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.684903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.685176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.685932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=ec66fef9dc274223b34659e369063ced) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:05.686240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.687753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.694350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:05.705812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 440/870] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:05 2022] GET /identity/v3/role_assignments?user.id=ec66fef9dc274223b34659e369063ced&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:05.737905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.738578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.738923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.739258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.740501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.742098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:05.752084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:05.753950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:05.756499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 431/871] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 13:23:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:05.768746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:05.874956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:05.894020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d03e0-c2fe-47d2-b3e7-c04e36895f8b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:05.933059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:05.933687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:05.933966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:05.934223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:05.934879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:05.935286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:05.936766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bdd8c-85cf-4a0e-9053-8ecd367555d7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:05.942660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 441/872] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:05.952498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:06.159648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a08463-b420-42ad-828a-9be71cfa91cf None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:06.228590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:06.229606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:06.230116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:06.230612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:06.231853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:06.232480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:06.235121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f19a81-9fc2-4d14-81fc-a5e3a29bacaf None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:06.245016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 432/873] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:05 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:06.618983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6980c5-5ee5-40e3-a96c-6ad65584f5e2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:06.619844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6980c5-5ee5-40e3-a96c-6ad65584f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:06.619844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6980c5-5ee5-40e3-a96c-6ad65584f5e2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:06.621142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 442/874] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:06.626845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:06.627182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:06.627432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:06.648129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:06.649355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:06.681356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:06.742832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 433/875] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:06 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:06.766600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6980c5-5ee5-40e3-a96c-6ad65584f5e2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:06.879699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6980c5-5ee5-40e3-a96c-6ad65584f5e2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:06.918008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:06.918720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:06.918999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:06.919310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:06.932289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:06.937225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:06.939477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:06.941429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 443/876] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:06.945531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.096964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fcb3d3-ddf5-4983-9783-773cbbe4e05d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.137497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.138045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.138325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.138598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.139317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:07.139625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.141031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4531e-6e8a-4cc0-97aa-8d382a504485 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.151060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 434/877] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:07.156826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.171646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:07.267497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e229b1-ca52-4035-96eb-3509e9cad608 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.304589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.305649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.306151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.306629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.311662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64686a-9035-4e2a-8101-c294e5553898 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.311957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64686a-9035-4e2a-8101-c294e5553898 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.312209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b64686a-9035-4e2a-8101-c294e5553898 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.313842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 435/878] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:07.319311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.319610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.319867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.321750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.323529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.339546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:07.339779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:07.340696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:07.341600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:07.341600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:07.341600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:07.341778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:07.343902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 444/879] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:07.349563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.368193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:07.445636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:07.451181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 436/880] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:07 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:07.486583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.509018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748e382-e227-43f4-803d-1c4f655e1dae None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.546985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.548198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.548533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.548865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.549587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:07.549958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.551560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.571121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 445/881] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:07.573093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.623145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3fed2-9942-4d64-abc5-cb5575c09127 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.682666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.683895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.684445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.684961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.686174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.688036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:07.700612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:07.701743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:07.702600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:07.707509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 437/882] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:07.712147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.714915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5377e77-9db4-4ce7-8af2-1b218c3869ba None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.745610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:07.747356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:07.762284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.763889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.764560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.765136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.774996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.776943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:07.791615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:07.792654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:07.793607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:07.801829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 446/883] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:07.802758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.912996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43617c0-e17b-41c2-a93c-372d541b6bac None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.946022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3435f239-9830-46dc-a6aa-a4bb46159c67 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:07.958843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.959945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.960403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.961089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.962406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:07.962892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.965031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:07.971944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 438/884] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:07.978537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:07.993737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:07.994453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:07.994734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:07.995003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:07.996033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:07.996292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:07.997958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.031105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 447/885] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:08.034384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.090029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.122136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.144160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6979ef4-92c2-4c7e-af61-ca9e492fb2e8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.171221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21709d5f-62c9-442e-9200-2001e3b67f7a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.190564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.191944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.192393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.192819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.193630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:08.194132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.196510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.204614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 439/886] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:08.211443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.217203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.217455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.217722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.217913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.300684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.302028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b1adc-4616-4f0c-bb4a-2107631bccec None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.332941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de0301-15f8-49e1-82b4-508a0d13ec70 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.346645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.370547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.371583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.372082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.372574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.373498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] RBAC: Adding query filter params (interface=public, service_id=3b6e135cb3544342ac05ea326a97b3ac, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:08.374002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.378803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.391183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 440/887] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/endpoints?service_id=3b6e135cb3544342ac05ea326a97b3ac&interface=public®ion_id=RegionOne => generated 194 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:08.400408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.420647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 448/888] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:08.422144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634422aa-6d8f-4acd-a86c-d624d7ed54a8 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.422421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634422aa-6d8f-4acd-a86c-d624d7ed54a8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.422694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634422aa-6d8f-4acd-a86c-d624d7ed54a8 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.424333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 449/889] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:08.425042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634422aa-6d8f-4acd-a86c-d624d7ed54a8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.429590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.527566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca1542-f3d6-49c1-8257-cbb91163c071 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.556393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634422aa-6d8f-4acd-a86c-d624d7ed54a8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.567868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.568479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.568780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.569079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.569692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.571117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff4de20-1fe6-4653-9e03-892288b6f3be None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.579591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 441/890] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/services => generated 1011 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:23:08.579591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.579591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.579591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.617450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:08.619329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:08.625403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.626160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.626498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.626826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.638846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.642747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.645799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:08.668429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:08.669664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:08.670779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:08.670779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:08.670779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:08.673864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 450/891] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:08.683862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.723057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 442/892] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:08.749388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.767013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:08.820315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec8bcf-a2ca-44e5-b741-e74ca661c096 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.866675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.867322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.867596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.867855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.869035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:08.869488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.871267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b49aeaa-a437-43fb-aaf3-32ada981f753 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.881877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a08d2-f6fa-48eb-8a82-71e6e083c3a9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:08.886408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 451/893] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:08.888803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ce785-41d7-4de1-9d6a-a1808d4b1926 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.889520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ce785-41d7-4de1-9d6a-a1808d4b1926 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.890193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ce785-41d7-4de1-9d6a-a1808d4b1926 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.893076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 452/894] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:08.900876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ce785-41d7-4de1-9d6a-a1808d4b1926 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:08.924719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:08.925577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.926444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.926751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.927431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:08.929099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-2cabfcf0-4718-440b-9d6c-c373f46990bc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:08.939520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:08.941705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:08.943456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 443/895] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:08.945105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:08.945105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:08.945105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:08.973280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:08.974514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:08.977142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:09.057182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:09.065050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 444/896] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:08 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:09.065702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:09.076122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ce785-41d7-4de1-9d6a-a1808d4b1926 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.119240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.119903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.120170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.120418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.130844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.132581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:09.146159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:09.153536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 453/897] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:09.154675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:09.222764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f7086-8de3-4791-9fe8-d6047e464b42 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.270672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.271383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.271688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.272052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.272904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:09.273705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.274856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.286258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 445/898] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:09.286258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:09.315517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81e688-e386-45dc-9592-6d471bab1478 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.331957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:09.369467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.370417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.370876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.371373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.372288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.373835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.400419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:09.402694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 454/899] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:09 2022] POST /identity/v3/services => generated 269 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:09.404489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:09.471209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bd8809-13e8-4f76-9153-5e28987cbdb8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.566897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.568342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.568844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.569341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.571039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:09.571697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.575223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.607047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 446/900] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:09.607250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24fbca-f2bd-4adb-a4c9-33d439ddacbd None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.613272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:09.680348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.681412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.681882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.682347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.684170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:09.684170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.686318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992889f-5ea8-463d-84f1-0ba81d032e47 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.694889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 455/901] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:09 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:09.765456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a96083-8ea6-4194-939b-08e3cb2bd5cb None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:09.792571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:09.810503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:09.811521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:09.811952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:09.812383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:09.813342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] RBAC: Adding query filter params (role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=6afd5333a03e4e098460fbbeef927d1b) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:09.813819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:09.815597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280c95e6-b05a-4c65-90b8-8ccb60effa17 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:09.823274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 447/902] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:09 2022] GET /identity/v3/role_assignments?user.id=6afd5333a03e4e098460fbbeef927d1b&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:10.997830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19320-031a-4e0a-9637-068f4a150965 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:10.998634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19320-031a-4e0a-9637-068f4a150965 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:10.998634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e19320-031a-4e0a-9637-068f4a150965 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.000081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 456/903] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:11.006950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.007499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.008287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.056070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:11.057465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:11.143317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 448/904] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:11.178776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e19320-031a-4e0a-9637-068f4a150965 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:11.276871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:11.420927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e19320-031a-4e0a-9637-068f4a150965 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:11.488638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:11.491591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.492216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.493166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.494408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:11.496967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:11.514605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:11.515717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 08 13:23:11.516563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:11.519939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 457/905] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:11.529686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:11.676696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d98c7b-fb44-483c-a8c5-cb7ceab668ac None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:11.701555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:11.736732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:11.737375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.737655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.737927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.738583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:11.738888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:11.740321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf052066-5445-4bbd-b135-2b668173eab0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:11.749678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 449/906] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:11.758086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:11.795110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b994-cf22-479e-bc87-fc74cea79789 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.795296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b994-cf22-479e-bc87-fc74cea79789 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.795660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3b994-cf22-479e-bc87-fc74cea79789 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.797448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 450/907] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:11.803373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.803662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.804099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.828172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:11.829471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:11.896999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:11.901517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb3c0e-727c-455b-bc07-4e03c8a0e486 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:11.932193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 451/908] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:11 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:11.953203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:11.954237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:11.954712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:11.955236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:11.956087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:11.956598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:11.958569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:11.959458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:11.966172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 458/909] 10.222.0.33 () {56 vars in 1114 bytes} [Mon Aug 8 13:23:11 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:11.972387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:12.087358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b7162-bbc5-424a-ab04-4249787b0fd4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.090313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd6b3a-3e1a-4ce8-9512-c36092a3547b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.102598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:12.128357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.129601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.130226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.130905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.132141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.133265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.134012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.134399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.134737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.134999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.135498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.136952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:12.147716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:12.148873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:12.149770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:12.151170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 452/910] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 13:23:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:12.157793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:12.174531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 459/911] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:11 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:12.300375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83470a42-fcee-49aa-8a6d-a18697ac31ee None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.352283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.353318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.353813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.354294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.355231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:12.355782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.357427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.367383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 453/912] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:12 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:12.370610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:12.398123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:12.492038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fc1341-f25c-4a5e-823a-3788d83a547d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.531214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.532241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.532739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.533209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.534160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:12.534673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.536327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.542824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 460/913] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:12 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:12.548767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:12.584867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:12.665068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f336f71-0b31-4baf-91eb-a3bbca3676a3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.703748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.704679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.705115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.705533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.706454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] RBAC: Adding query filter params (interface=public, service_id=fd5a9b0a492c4ad0a722d687dff4b9a3, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:12.706917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.708522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce50ab-526a-4886-8b43-6bf00feaeb9f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.719215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 454/914] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:23:12 2022] GET /identity/v3/endpoints?service_id=fd5a9b0a492c4ad0a722d687dff4b9a3&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:12.721734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:12.803053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e7-0dc0-4ebe-862b-d4dceec04ac3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.803536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e7-0dc0-4ebe-862b-d4dceec04ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.803983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e7-0dc0-4ebe-862b-d4dceec04ac3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.806812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 455/915] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:12.815482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25c933-0567-4f9a-9dbc-3d89ce540147 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.815963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25c933-0567-4f9a-9dbc-3d89ce540147 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.816434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25c933-0567-4f9a-9dbc-3d89ce540147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.818101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:12.838048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4e1a5-f3b6-4e96-8e4f-a0f204117d31 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:12.852699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6c25c933-0567-4f9a-9dbc-3d89ce540147 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:12.855329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6c25c933-0567-4f9a-9dbc-3d89ce540147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:12.910725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:12.911951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:12.912429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:12.912888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:12.913890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:12.917753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:12.930058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 461/916] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:12 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:23:12.987091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 456/917] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:12 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:13.025193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:13.043906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78fb6b-c755-4253-80e4-704d5a49cf34 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.044478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78fb6b-c755-4253-80e4-704d5a49cf34 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.044975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78fb6b-c755-4253-80e4-704d5a49cf34 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.048156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 457/918] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:13.053518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.053829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.054059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.076406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:13.077797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:13.111858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:13.186743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 458/919] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:13.200702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d9cc14-322d-4410-bb6b-a79fe67d619c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:13.211571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:13.278739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:13.280355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.280857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.281345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.296708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:13.304913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:13.305947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:13.306823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:13.313878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 462/920] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:13.316485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:13.329502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:13.375060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5c154e-a837-4708-a81c-5cadd7a29320 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:13.434627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1911e-3197-4c21-8e94-fdc81235291a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:13.442231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:13.443609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.444373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.445150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.446624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:13.449372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:13.475292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:13.475971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.476248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.476522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.483126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:13.483126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:13.483126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:13.484415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 459/921] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:13 2022] POST /identity/v3/services => generated 241 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:13.494851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 463/922] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:13.497090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:13.522785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:13.658018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f52ad3-bcba-4f2a-ab71-be4d4a36fe1b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:13.699247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:13.699842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.700128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.700461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.720009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:13.722827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:13.752209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:13.753299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:13.754211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:13.754211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:13.754211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:13.754816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 460/923] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:13.762738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:13.879821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:13.916587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487f4c-54fe-4620-9414-c81a2e46a390 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:13.953079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:13.953622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:13.953902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:13.954184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:13.954835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:13.958422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:13.958422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:13.971062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 464/924] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:13.977408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.050719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:14.102680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af481d61-1a3b-4eb5-aab8-055a90851e12 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.150300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:14.151134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.151508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.151787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.162864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:14.164415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:14.181141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:14.182180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:14.183154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 461/925] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:23:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:14.190610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.350903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbac6d-52e5-420d-9eae-51f5941862e1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.403965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:14.411832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:14.412898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.413394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.413915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.414867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:14.415443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:14.417240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:14.446447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 465/926] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:23:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:14.455014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.589525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083964a0-0db8-49af-9046-73a72e823a67 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.615005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:14.626627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:14.627621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.628343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.628615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.641293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:14.642787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e97ebe69-e985-4bf1-93f7-8381895bc990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:14.658251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:14.659260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:14.660154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:14.660154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:14.660154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:14.660154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 462/927] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:14.665601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.684512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb326d-c8ad-4bd5-8ce4-5655fa02ec4a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.684974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb326d-c8ad-4bd5-8ce4-5655fa02ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.685357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb326d-c8ad-4bd5-8ce4-5655fa02ec4a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.687416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 463/928] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:14.693791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.694091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.694384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.718923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:14.720787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:14.784032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19658c1f-3d4a-494f-ad3e-84273b765bcc None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.823985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:14.824666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:14.824951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:14.825332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:14.826006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:14.826309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:14.827751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:14.839452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 464/929] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:14 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:14.841628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 466/930] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:14.847744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.864113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:14.927542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:14.984990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81272e-8f2e-49ef-88e0-795785759efd None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.991678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df87a-ac37-4e09-a6f5-3b1dffe53066 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:14.998815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:15.037460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.038376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.038806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.039465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.040529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.042687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.045103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.046224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.046407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING keystone.server.flask.common [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 13:23:15.046726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.047241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.060329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.063168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.070291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.sql.core [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 13:23:15.070291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 13:23:15.070291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [parameters: {'id': '151ecf5b9e0541d48f5ebe13a31bfef3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 13:23:15.070291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80188) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-dae73885-a9f6-422f-b9fc-1d64b922ba4d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 151ecf5b9e0541d48f5ebe13a31bfef3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 151ecf5b9e0541d48f5ebe13a31bfef3. Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:23:15.078853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result.read() Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 08 13:23:15.079924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application yield resource Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise self.value Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application session.commit() Aug 08 13:23:15.081115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.session.flush() Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application compat.raise_( Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 13:23:15.082129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 13:23:15.083316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application result.read() Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:23:15.084163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [parameters: {'id': '151ecf5b9e0541d48f5ebe13a31bfef3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.085066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:15.085989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 151ecf5b9e0541d48f5ebe13a31bfef3. Aug 08 13:23:15.092417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:15.093871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:15.099917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:15.102554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 465/931] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:15.103920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 467/932] 10.222.0.33 () {60 vars in 1140 bytes} [Mon Aug 8 13:23:14 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:15.104067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.104220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69839c0-4b3b-4687-87c7-063f4332456f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.104220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69839c0-4b3b-4687-87c7-063f4332456f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.104220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69839c0-4b3b-4687-87c7-063f4332456f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.107216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 468/933] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:15.108253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69839c0-4b3b-4687-87c7-063f4332456f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.234761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69839c0-4b3b-4687-87c7-063f4332456f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:15.237674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:15.277403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.278149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.278491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.278825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.286001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8f7390-3840-44a8-90d2-84b8d30e61a3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:15.287973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.289782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-dbd5f6e4-055b-4684-ba04-f810af3fb50a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:15.302182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:15.312278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 469/934] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.314024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.315932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.333347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:15.336284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:15.349443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.350346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.350694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.351037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.351866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:15.352351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.353827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.376817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 466/935] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:15.376817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.426013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:15.451530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 470/936] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:15.453722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.535515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611eeec-25c6-4e04-adec-4dea182c0a25 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:15.583188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.584422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.584962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.585515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.586781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:15.587120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.589037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.599491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:15.613423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb2cbf1-627f-4e70-9055-df6530559637 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:15.615916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 467/937] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:15.616656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.684380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.686154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.686839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.687621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/6afd5333a03e4e098460fbbeef927d1b/roles/f7a2a9d4dbb24ce4bf02ee811622cf3b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.692164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=6afd5333a03e4e098460fbbeef927d1b, role_id=f7a2a9d4dbb24ce4bf02ee811622cf3b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.695599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0ae80e-d34a-445e-807d-fcc17d84f7c7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.726455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 471/938] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:23:15 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/6afd5333a03e4e098460fbbeef927d1b/roles/f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:23:15.762877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:15.789834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8b987-c2a1-4dac-9d09-8926fb9617dc None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:15.814304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a964-c467-4377-9529-2b8f6fc9d40a None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.814582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a964-c467-4377-9529-2b8f6fc9d40a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.814953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4a964-c467-4377-9529-2b8f6fc9d40a None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.816702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 472/939] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:15.822503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.822855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.823252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.847293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:15.848546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:15.862013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:15.862746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:15.863478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:15.863999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:15.864770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:15.867109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:15.879137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:15.881292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:15.883684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 468/940] 10.222.0.33 () {56 vars in 1132 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:15.890290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:15.956587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 473/941] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:15 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:15.983478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.069228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702445e8-b091-4e4e-ae62-a4bb5fa63cf5 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.131297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:16.141096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.142949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.143617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.144166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.145442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:16.146226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.149304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.158332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 469/942] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:16.165476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684ade-49d4-4643-86c1-ce3dacd4d8d0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.166734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.193459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:16.239051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.240637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.241249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.241997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.243300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.246218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:16.265626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:16.266863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:16.267891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:16.271721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 474/943] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:16.278078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.340270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4540ed8-725f-42b9-8c45-4cf33e715072 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.404401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.405772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.406253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.406638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.407651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:16.407899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cae47-f4a1-42d5-83e4-1831d8524b0a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.408183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.410949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.423581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 470/944] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:16.431323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.455363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.456111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.456111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.456111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.456950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:16.457316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.458796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.470654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 475/945] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:16.472196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.491967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:16.604851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f49b3d-63f9-4ee2-b25b-d3fda75b1213 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.605167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:16.640429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9476b3-ea85-4b3a-b338-64473f780888 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.656679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.657310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.657587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.657855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.658498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:16.658797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.660310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.672162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 476/946] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:16.678183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:16.696590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.697218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.697496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.697766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.699120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.700523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:16.746038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 471/947] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:16 2022] POST /identity/v3/endpoints => generated 347 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:16.863599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:16.902094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0ce08-130c-4cb4-b8a3-3be8853dfb79 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:16.982860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:16.983958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:16.984441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:16.984918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:16.986245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] RBAC: Adding query filter params (interface=public, service_id=8952c6175dca4b68b023b8cd2f51afa7, region_id=RegionOne) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:16.986792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:16.989520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b850f-b564-4c6e-b2fb-353871904407 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:17.000733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 477/948] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:23:16 2022] GET /identity/v3/endpoints?service_id=8952c6175dca4b68b023b8cd2f51afa7&interface=public®ion_id=RegionOne => generated 194 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:17.010497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:17.216869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42421c51-885a-4369-a407-687184e8ba3b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:17.287557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:17.288951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:17.289669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:17.290337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:17.291696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:17.294648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371b9e8-b32e-40e3-b489-c3d4cbaed92b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:17.297376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:17.313151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 472/949] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:17 2022] GET /identity/v3/services => generated 1497 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:23:18.121769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3c5a-88cf-432a-b2d1-24c07d7d6111 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.122456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3c5a-88cf-432a-b2d1-24c07d7d6111 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.122748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3eb3c5a-88cf-432a-b2d1-24c07d7d6111 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.125817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 478/950] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:18.135539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4778e61-1826-4449-96d2-ddebdd66e4a3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.136114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4778e61-1826-4449-96d2-ddebdd66e4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.136486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4778e61-1826-4449-96d2-ddebdd66e4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.168094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b4778e61-1826-4449-96d2-ddebdd66e4a3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:18.170122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b4778e61-1826-4449-96d2-ddebdd66e4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:18.290955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c0f9d-abf5-40bb-83c4-44cb61531d15 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.291340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c0f9d-abf5-40bb-83c4-44cb61531d15 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.291529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0c0f9d-abf5-40bb-83c4-44cb61531d15 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.292037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 473/951] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:18.292883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 479/952] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:18.298530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.298821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.299121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.325479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:18.327577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:18.334031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c0f9d-abf5-40bb-83c4-44cb61531d15 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:18.457030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 474/953] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:18.482021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:18.523751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0c0f9d-abf5-40bb-83c4-44cb61531d15 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:18.561925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:18.562983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.563520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.564124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.567467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:18.578147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:18.583536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:18.584693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:18.585605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:18.588585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 480/954] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:18.598009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:18.602522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:18.685411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba205651-a6ab-43ea-abeb-daafa51f0508 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:18.711649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a195353-d60d-47c0-b112-71c2391ce09d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:18.748610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:18.749226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.749488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.749749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.750417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:18.750714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:18.751910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:18.752195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:18.752989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.753451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.754138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.755522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:18.758594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:18.761443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING keystone.server.flask.common [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 08 13:23:18.766714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 481/955] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:18.767739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:18.782075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.sql.core [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 13:23:18.782075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 13:23:18.782075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [parameters: {'user_id': '67512235d51f4559a13c71d6be7f21e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 13:23:18.782075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80187) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application result.read() Aug 08 13:23:18.815274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:23:18.819147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application yield resource Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application next(self.gen) Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self.session.rollback() Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self.force_reraise() Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise self.value Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application session.commit() Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 08 13:23:18.821483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._prepare_impl() Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self.session.flush() Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._flush(objects) Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application compat.raise_( Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application flush_context.execute() Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rec.execute(self) Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 08 13:23:18.832344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 08 13:23:18.835935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application result = self._query(query) Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application conn.query(q) Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application result.read() Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 08 13:23:18.839106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [parameters: {'user_id': '67512235d51f4559a13c71d6be7f21e8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:18.842280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 08 13:23:18.845350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:18.848777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 475/956] 10.222.0.33 () {60 vars in 1131 bytes} [Mon Aug 8 13:23:18 2022] POST /identity/v3/users => generated 160 bytes in 337 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:18.850265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:18.850498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:18.879060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aea61e-961c-441a-a74b-3c397766b139 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:18.918018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:18.918670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:18.918957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:18.922197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:18.934247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:18.934247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:18.951679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:18.955397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:18.959050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:18.959050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:18.959050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:18.959050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 482/957] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:18.959050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.031967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97adb5-114a-4258-8878-949c113a712b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.054932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:19.074346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.074969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.075286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.075549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.084485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.086388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:19.101038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:19.102919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 08 13:23:19.106087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:19.107711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 476/958] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:23:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:19.108558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.177087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d5b83-4408-4355-b000-4370a6e805af None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.200123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:19.236312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf84431-d046-4b6e-9bbc-34a5c6a52650 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.246976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.248354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.249031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.249692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.251222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:19.252100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.254951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.278953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 483/959] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:19.291148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.291148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.291148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.291148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.291885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:19.291885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.292141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.292291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6c2a43-e231-41cf-9bf5-12899cdb8385 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.367342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 477/960] 10.222.0.33 () {56 vars in 1101 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:19.415749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090557e-cd64-4c57-80d9-f069cb243910 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.416057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090557e-cd64-4c57-80d9-f069cb243910 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.416325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9090557e-cd64-4c57-80d9-f069cb243910 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.419119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 478/961] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:19.424739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.424998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.425281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.445948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:19.447658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:19.488599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:19.508294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fa20a3-bca6-49b9-97ea-9af3aef94808 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.563511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 479/962] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:19.568138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.569399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.569955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.570435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.582663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.585554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.588623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:19.604004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:19.605871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 08 13:23:19.607761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:19.610191 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 484/963] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:19.611687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.671293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:19.762772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605d6baa-b132-4369-af92-5a4acc899c41 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.804750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd4565-fc09-48cb-9b4a-0e722a578b39 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:19.813863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.814587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.814865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.815245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.816146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.818107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:19.833174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:19.835833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:19.837468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 480/964] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:19.840923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.876021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:19.877977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:19.879143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:19.880070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:19.880664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:19.883100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:19.883620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:19.886433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:19.915498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 485/965] 10.222.0.33 () {56 vars in 1149 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:19.917169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:19.988544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf11cb-1b0d-462a-a25c-d05e9e80724a None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.035154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.035682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.035991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.036227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.036884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:20.037279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.038695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.049690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 481/966] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:20.051245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.071896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.087445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded9d40-4e12-4aaa-a263-a0ec5dc7212a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.129561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.130376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.130899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.131371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.190357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.198647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6476092-08c7-4847-9f24-7d2eec694cf9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.218117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.219652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.240926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.241949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.242404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.242857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.244043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] RBAC: Adding query filter params (type=compute) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:20.244567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.247298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab6734-4929-4922-9e89-55bfa5b097fe None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.256100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 482/967] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:20.318571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81623f4-f61e-4459-a02f-8aa90e3cba40 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.319313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81623f4-f61e-4459-a02f-8aa90e3cba40 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.319808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81623f4-f61e-4459-a02f-8aa90e3cba40 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.323600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 483/968] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:20.332284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.332745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.333214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.357223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.378656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:20.389019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:20.389353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 486/969] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:19 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:20.397387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.495186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 484/970] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:20.525834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.543599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9247d88-6c10-4b9f-b9c7-ef21d22e2e51 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.609990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.611061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.611554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.612055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.620624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.631610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.633486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.633672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576238fc-c02f-4ce7-9801-7595d95fd3d2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:20.652013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:20.655034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:20.656640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:20.656640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:20.656640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:20.657076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 487/971] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:20.661693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.690587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.691768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.692289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.692769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.693951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.696579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:20.710949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:20.712186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:20.713034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 485/972] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:20.720929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.723018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.808384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4478a11-7d41-4fb5-a5db-2fec4ef6ccda None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.846291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.846923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.847309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.847581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.848250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:20.848548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.850005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.859534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e60cb-3ba1-4806-a6d0-dbf58e7af5d9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:20.865732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 488/973] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:20.871918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.902490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:20.903397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:20.904108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:20.904405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:20.905319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:20.905624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:20.908014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:20.914297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 486/974] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:20.920154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:20.957123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.990177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:20.994221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc160d7-545a-4f70-a4cf-3992ecb136a2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:21.042013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:21.042805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:21.043300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:21.043740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:21.056062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:21.058467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:21.079435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:21.081325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:21.083315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 489/975] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:21.089577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:21.118785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34f2da4-754f-4a02-825e-b1db25c9c54b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:21.177249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:21.178258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:21.178705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:21.179186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:21.180203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:21.180694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:21.183098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:21.198976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 487/976] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:21.200977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:21.288650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedaffd1-4da1-42c1-8ae8-7405f05261c6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:21.346205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:21.347980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:21.348938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:21.349418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:21.350422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:21.351715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:21.352222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:21.354883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:21.379811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 490/977] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:21.381998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc314440-fae7-4d5f-90a9-2191a2d4751c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:21.388122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:21.425588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:21.447279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:21.448305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:21.448792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:21.449270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:21.450477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:21.453057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6094f953-392e-479c-a7c7-0a71da9f8635 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:21.492005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 488/978] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:21.537469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f27092-41db-481c-a0ef-8a76ab05a812 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:21.591787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:21.592665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:21.593086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:21.593511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:21.594558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] RBAC: Adding query filter params (role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=6afd5333a03e4e098460fbbeef927d1b) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:21.595052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:21.596905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d8392-3853-4cfd-bf66-9651651f2d05 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:21.605362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 491/979] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:21 2022] GET /identity/v3/role_assignments?user.id=6afd5333a03e4e098460fbbeef927d1b&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:22.458033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf16e1-64f9-445a-baa8-bc44f943862b None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:22.458804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf16e1-64f9-445a-baa8-bc44f943862b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:22.458804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf16e1-64f9-445a-baa8-bc44f943862b None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:22.460192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 489/980] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:22.465511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec27be-6adc-4d01-b70e-c2261a904683 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:22.466093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec27be-6adc-4d01-b70e-c2261a904683 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:22.466093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec27be-6adc-4d01-b70e-c2261a904683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:22.485729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-46ec27be-6adc-4d01-b70e-c2261a904683 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:22.488793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-46ec27be-6adc-4d01-b70e-c2261a904683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:22.567545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 492/981] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:22.589653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf16e1-64f9-445a-baa8-bc44f943862b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:22.710705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf16e1-64f9-445a-baa8-bc44f943862b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:22.750341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:22.750990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:22.751299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:22.751570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:22.752250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:22.753698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ddf97-7c9d-4bf7-83f9-e90e925c6959 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:22.767843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:22.776778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 490/982] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:22 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:23.668437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bcfa-e7b4-4996-a4e6-c21269fa6544 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:23.669004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bcfa-e7b4-4996-a4e6-c21269fa6544 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:23.669004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc30bcfa-e7b4-4996-a4e6-c21269fa6544 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:23.670350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 493/983] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:23.676069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:23.676793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:23.676793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:23.695980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:23.697230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:23.779139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 491/984] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:23 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:23.801607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bcfa-e7b4-4996-a4e6-c21269fa6544 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:23.814955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:23.919773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc30bcfa-e7b4-4996-a4e6-c21269fa6544 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:23.958059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:23.958712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:23.959087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:23.959375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:23.959973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:23.961377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:23.971045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:23.972146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:23.973113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:23.975483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 494/985] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:23.978451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.107448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50935f3-08b4-4760-a224-d21eb73e3143 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.136278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:24.151669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:24.152722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.153211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.153689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.154855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:24.155397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:24.158028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:24.165746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 492/986] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:24.172183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.268664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:24.309523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d178dc8-b170-45c9-aaae-902a611d0ece None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.349443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:24.350106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.350378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.350664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.351324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:24.351621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:24.353009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2780b5-b0ef-4646-95b5-f561ada62f38 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:24.358710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 495/987] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:24.365161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.419296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7056c-abce-4bb4-8fd5-b6c22167e155 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.419584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7056c-abce-4bb4-8fd5-b6c22167e155 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.419858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb7056c-abce-4bb4-8fd5-b6c22167e155 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.421523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 496/988] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:24.427336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.427532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.427918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.450859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:24.453404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:24.503421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09462c1f-91c7-422e-b595-ea9e16b50590 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.548107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:24.548841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.549267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.549546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.550387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] RBAC: Adding query filter params (interface=admin, service_id=8952c6175dca4b68b023b8cd2f51afa7, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:24.550698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:24.552159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:24.555600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 497/989] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:24 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:24.563194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 493/990] 10.222.0.33 () {56 vars in 1249 bytes} [Mon Aug 8 13:23:24 2022] GET /identity/v3/endpoints?service_id=8952c6175dca4b68b023b8cd2f51afa7&interface=admin®ion_id=RegionOne => generated 193 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:24.567698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.578089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.642295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:24.678806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:24.700787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5919e39-cd67-45db-b83f-377f21be3494 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.739603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d1fc0-1b52-4994-b755-f9030d2bdb53 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.751571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:24.752672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.753148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.753624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] PATH_INFO: `/v3/services/image` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.754763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:24.757707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:24.768184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:24.778900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:24.780660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 494/991] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 13:23:24 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:24.782694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:24.818818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:24.819904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:24.820436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:24.820844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:24.821554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:24.823801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:24.835578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 498/992] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:24 2022] GET /identity/v3/services => generated 1727 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:23:24.962130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1210e88d-ca85-4987-8029-9957cf2c7ff6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:24.993914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:25.028700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:25.029641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.030059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.030471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.031573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] RBAC: Adding query filter params (name=image) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:25.032029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:25.034352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2b8cd4-af1d-4dbc-b806-17d4db492313 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:25.043945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 495/993] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:25.053253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:25.196415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58c6c-82af-4688-8285-a8d62ee211c1 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.196685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58c6c-82af-4688-8285-a8d62ee211c1 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.197026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c58c6c-82af-4688-8285-a8d62ee211c1 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.199128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 496/994] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:25.207390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.207897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.208220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.226764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18145ed-4549-481a-92f3-c2a60e03da50 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:25.238008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:25.240200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:25.267904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:25.276034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:25.276821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.277227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.277753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.278812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] RBAC: Adding query filter params (type=image) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:25.279295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:25.280966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:25.287591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 499/995] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:25.347138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 497/996] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:25 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:25.368639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:25.514680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb004e69-7480-4993-a41c-817395742d7e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:25.577485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:25.578550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.579029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.579555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.580602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:25.583235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:25.600589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:25.602592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:25.604310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 500/997] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:25 2022] GET /identity/v3/services/compute => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:25.611762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:25.796192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e498a3-59fa-419e-b4d8-03f9df42eb17 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:25.817471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:25.862551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:25.863639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:25.864134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:25.864605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:25.865839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:25.866350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:25.868915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:25.878152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 498/998] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:25 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:25.886231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:25.996407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:26.077173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915138c8-ec46-428d-9880-532b5666dc2d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:26.143893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:26.145003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:26.145514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:26.146014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:26.147396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] RBAC: Adding query filter params (type=compute) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:26.147951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:26.150816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:26.161753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 501/999] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:25 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:26.171614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:26.360356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb091226-9353-4779-8636-2cc2f8380e18 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:26.432426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:26.433554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:26.434034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:26.434504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:26.435876 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] RBAC: Adding query filter params (interface=public, service_id=7a8186f6e2cf41709e3c2b8b12a32777, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:26.436410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:26.439179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae23fa-c9d0-449a-8d1a-09c117e5037c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:26.455448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 499/1000] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:23:26 2022] GET /identity/v3/endpoints?service_id=7a8186f6e2cf41709e3c2b8b12a32777&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:26.459730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:26.585126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187ff0f6-0587-458a-a9b7-75a01d4ab1df None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:26.613892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:26.626912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:26.627570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:26.627846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:26.628143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:26.628731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:26.630118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f439312-c80f-4889-ab5d-e8eab3c1b2fb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:26.637941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 502/1001] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:26 2022] GET /identity/v3/services => generated 1727 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:23:27.070398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f261-e48a-43a8-993c-efa470b25c46 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.071127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f261-e48a-43a8-993c-efa470b25c46 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.071127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22f261-e48a-43a8-993c-efa470b25c46 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.072339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 500/1002] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:27.079375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.079906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.080329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.112530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:27.114647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:27.234288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 503/1003] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 08 13:23:27.272955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f261-e48a-43a8-993c-efa470b25c46 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:27.346299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:27.446402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22f261-e48a-43a8-993c-efa470b25c46 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:27.496035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:27.497329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.497932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.498413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.499439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:27.501732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:27.513017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:27.514101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 08 13:23:27.514957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:27.517397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 501/1004] 10.222.0.33 () {56 vars in 1126 bytes} [Mon Aug 8 13:23:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:27.525732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:27.662548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaba699-3c7e-4f06-9487-8a233395105d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:27.694964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:27.722659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:27.723412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.723809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.724086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.725207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:27.725651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:27.727900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa64445-01f0-4e9e-9c57-7ef992a32da5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:27.737602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 504/1005] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:27.746964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:27.876619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0191-0aa0-41b9-b9ab-26038a6829dc None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.877120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0191-0aa0-41b9-b9ab-26038a6829dc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.877481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ae0191-0aa0-41b9-b9ab-26038a6829dc None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.880044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 505/1006] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:27.888280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.888794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.889018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.918294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:27.920082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:27.946964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e9907-9b74-4843-a8d6-baa4cdeb7927 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:27.949899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:27.994731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:27.995382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:27.995684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:27.995956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:27.996604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:27.996905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:27.998332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:28.006438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 502/1007] 10.222.0.33 () {56 vars in 1122 bytes} [Mon Aug 8 13:23:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:28.012757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:28.027148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 506/1008] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:28.050622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:28.161617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47859da-0dab-4dbb-aeae-07ac502f507d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:28.221357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:28.222449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:28.222959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:28.223505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:28.224743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:28.227345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ce8bca-70ec-4e05-a0b2-3d446a168ca4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:28.237883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d88bcb-8731-404e-8ed9-89f72e57776f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:28.257631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 503/1009] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:28.272243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:28.312382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:28.313334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:28.313745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:28.314211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:28.315072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:28.316955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1991b0b3-4f8f-473e-a7c0-9e25701cf7b3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:28.347155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 507/1010] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:28 2022] POST /identity/v3/services => generated 242 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:28.776243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf725-c414-4a08-9523-6777ebdeb8ac None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:28.776557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf725-c414-4a08-9523-6777ebdeb8ac None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:28.776891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fcf725-c414-4a08-9523-6777ebdeb8ac None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:28.778387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 504/1011] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:28.783899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2177b87-0706-41d7-a027-3154de934291 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:28.784276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2177b87-0706-41d7-a027-3154de934291 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:28.784518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2177b87-0706-41d7-a027-3154de934291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:28.803566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e2177b87-0706-41d7-a027-3154de934291 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:28.804788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e2177b87-0706-41d7-a027-3154de934291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:28.890723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 508/1012] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:28 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:28.913780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf725-c414-4a08-9523-6777ebdeb8ac None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:28.945177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:29.032068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fcf725-c414-4a08-9523-6777ebdeb8ac None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:29.071434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:29.072086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:29.072351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:29.072624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:29.073295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:29.074709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:29.084947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:29.086896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 08 13:23:29.088532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:29.091639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 505/1013] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:29.092523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2177b87-0706-41d7-a027-3154de934291 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:29.203794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:29.213162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2177b87-0706-41d7-a027-3154de934291 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:29.251103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:29.252117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:29.252117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:29.252362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:29.252900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:29.253201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:29.254590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:29.260596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 509/1014] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:29.266886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:29.370551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:29.390443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9190bf09-4f65-4dff-9014-ce21b1f9ef5b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:29.429727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:29.430352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:29.430631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:29.430901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:29.431593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:29.431896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:29.433297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cca984-c909-498f-bc53-28d2433b0f5a None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:29.439514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 506/1015] 10.222.0.33 () {56 vars in 1112 bytes} [Mon Aug 8 13:23:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:29.445346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:29.569779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec16c9-8fc3-4cce-9924-3d607ad55544 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:29.610719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:29.611368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:29.611644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:29.611910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:29.612600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:29.614001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f4ef6-a966-4318-945f-dceac3aa8832 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:29.647227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 510/1016] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:29 2022] POST /identity/v3/endpoints => generated 334 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:31.124358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b718-cc7e-474b-8891-d6d889722a1d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.125068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b718-cc7e-474b-8891-d6d889722a1d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.125068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd66b718-cc7e-474b-8891-d6d889722a1d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.126713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 507/1017] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:31.132715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.133224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.133338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.153168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:31.154475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:31.168325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:31.169122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17ae0-abaf-4b03-bb9b-436c7192aace None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.169618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17ae0-abaf-4b03-bb9b-436c7192aace None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.170354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17ae0-abaf-4b03-bb9b-436c7192aace None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.172971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 508/1018] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:31.183241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.184190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.184190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.217419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:31.219598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:31.252327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 511/1019] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:31.275211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:31.339184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 509/1020] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:31 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:31.363013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:31.450133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e60d53f-f999-4354-8af2-349db8fe9150 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:31.473360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:31.496267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddf68-6096-48f8-b38a-7d9116784621 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:31.502925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:31.503756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.504037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.504310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] PATH_INFO: `/v3/services/image` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.504919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:31.506370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:31.522086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:31.523251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:31.524121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:31.524872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 512/1021] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 13:23:31 2022] GET /identity/v3/services/image => generated 86 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:31.533925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:31.540832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:31.561450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:31.562583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.563097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.563798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.565708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:31.568627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:31.574615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 510/1022] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:31.580526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:31.706154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adae76d-fc2d-4c60-8b6d-023db51eab16 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:31.748850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:31.749668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.749941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.750413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.750981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] RBAC: Adding query filter params (name=image) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:31.751312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:31.752892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:31.759148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 513/1023] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:31.764191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b3fac-1f39-4e1b-93c6-97e547b41850 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:31.765096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:31.808443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:31.809077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:31.809353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:31.809635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:31.810310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:31.811845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:31.850744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 511/1024] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:31 2022] POST /identity/v3/projects => generated 311 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:31.886978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:31.954092 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6ce7-c66d-4aab-9f85-4365f224ee7a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.021254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.022073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.022356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.022626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.023342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] RBAC: Adding query filter params (type=image) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:32.023643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:32.025092 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbaa9d-db34-4c13-af1f-c5e692ece61b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:32.031934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 514/1025] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:32.038018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:32.043511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:32.055580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa826b3-b835-4d45-a5ab-95e219e16c33 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.055911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa826b3-b835-4d45-a5ab-95e219e16c33 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.056188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa826b3-b835-4d45-a5ab-95e219e16c33 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.057736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 515/1026] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:32.063455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.063736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.064009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.084533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:32.085678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:32.170398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 516/1027] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:32 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:32.172225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304026b7-3b18-46ec-943a-529b9c7c144c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.193906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:32.216954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.217674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.218256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.218256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.219134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] RBAC: Adding query filter params (interface=public, service_id=9b46e32ab12949cf901aec57fc440e70, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:32.219584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:32.221311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:32.228421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 512/1028] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/endpoints?service_id=9b46e32ab12949cf901aec57fc440e70&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:32.235017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:32.287785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:32.347966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:32.362042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d9c74d-514e-4294-959c-fea25c86dee5 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.409814 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59024b68-34a3-49ff-90ef-c19d5573f564 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.421115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.422243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.422614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.422957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.436528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:32.441797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:32.444057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:32.446114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:32.449198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 517/1029] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:32.450248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:32.464001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.464725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.465074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.465355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.466436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:32.469070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:32.482090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 513/1030] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/services => generated 1957 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:23:32.592046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255a80-8ff7-4281-be12-a49efaa8156a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.625362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:32.643733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.644509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.644790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.645095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.646044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:32.646354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:32.648268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:32.659762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 518/1031] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:32.666182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:32.820429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123735d6-6aa1-4448-93d9-8136e3e2839b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:32.859187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:32.866498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:32.867258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:32.867498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:32.867751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:32.882747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:32.884366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:32.901214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:32.902960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:32.904996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:32.904996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:32.904996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:32.907202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 514/1032] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:32.909093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:33.036928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70c896-9ed0-4052-ab2c-c3d58988930c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:33.076730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:33.077358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:33.077675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:33.077986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:33.078650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:33.078951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:33.080378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:33.095147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 519/1033] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:33.101903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:33.227452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7db0de6-0e4d-4394-828c-0a82f710838e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:33.266308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:33.266948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:33.267261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:33.267533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:33.276412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:33.277936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:33.290595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:33.300055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:33.301966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 515/1034] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:23:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:33.307069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:33.387344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:33.465217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb5ea9-ee90-4cac-ab52-705b15e7144a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:33.514355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934153-398d-4346-bf66-858b46c0261d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:33.515045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934153-398d-4346-bf66-858b46c0261d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:33.515424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934153-398d-4346-bf66-858b46c0261d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:33.515696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934153-398d-4346-bf66-858b46c0261d None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:33.516719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934153-398d-4346-bf66-858b46c0261d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:33.517043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934153-398d-4346-bf66-858b46c0261d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:33.518775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934153-398d-4346-bf66-858b46c0261d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:33.543984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 520/1035] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:23:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:33.551832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:33.624895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:33.702560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d84cc5-a185-431f-a002-6b76ceabfc06 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:33.759545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:33.760622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:33.761101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:33.761587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:33.878141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:33.880960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:33.955450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.007375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe662a-99c7-4da1-b3f6-6dc1398735ff None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.007990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe662a-99c7-4da1-b3f6-6dc1398735ff None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.008551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe662a-99c7-4da1-b3f6-6dc1398735ff None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.011047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 521/1036] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:34.019915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.020390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.020803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.034506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 516/1037] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:33 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:23:34.043874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.045088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:34.046472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:34.127411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 522/1038] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:34.152047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.201531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.206203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8dd7a-4fd8-47a3-8ac1-50c8ce73e4a3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.272341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.273336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.273336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.273763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.286657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570ca29-57a6-4ad8-8257-1fc3bfbdf70b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.289062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.292163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.297260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:34.321914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:34.322979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:34.329269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:34.329269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:34.329269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:34.329269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 517/1039] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:34.329853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.333733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.334618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.335122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.335600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.337559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.340347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.343156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 523/1040] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:34.352135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.458850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89557c-9d8d-4345-9d24-afe79039bc99 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.497614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.498419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.498798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.499184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.500090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:34.500448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.502049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.505746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.511827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc29c1d6-b1bb-47e8-b8cb-f24bd93b9659 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.524002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 518/1041] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:34.525653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.564459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.565218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.565586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.566022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.566808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.568515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f3663c-efa2-49c8-b098-faebe614c56e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.592132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.607401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 524/1042] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:23:34 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:23:34.607401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860a2-3284-46d9-9728-884fcae37d56 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.607401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860a2-3284-46d9-9728-884fcae37d56 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.607401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2860a2-3284-46d9-9728-884fcae37d56 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.607401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 525/1043] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:34.615112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.615112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.615112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.633222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:34.634937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:34.667744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9630c5-a32c-4776-8be2-63d81a71fc27 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.710149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.710786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.711149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.711397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.719884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.721364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:34.736550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:34.737610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:34.738863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:34.743118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 519/1044] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:34.743993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.755264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 526/1045] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:34.776060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.878425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375aefff-091f-44e7-933b-c8b476865366 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.893299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.898975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:34.914884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce74f7b-b107-4340-87c0-d5c40cc95de3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:34.922316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.922949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.923254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.923524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.924236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:34.924532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.926010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.939550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 520/1046] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:34.945978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:34.963279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:34.963788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:34.964218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:34.964326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] PATH_INFO: `/v3/services/image` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:34.964981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:34.966410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:34.976100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:34.978053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:34.979317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 527/1047] 10.222.0.33 () {56 vars in 1105 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:34.983835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:35.102870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e128fde-6495-4e1f-9edb-e3511c410aea None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:35.143481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd7c08-0d73-4284-a011-45d770828463 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:35.161650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:35.162278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:35.162536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:35.162856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:35.163591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] RBAC: Adding query filter params (role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:35.164032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:35.165354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:35.174345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 521/1048] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:35.194530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:35.195661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:35.196168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:35.196647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:35.197861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] RBAC: Adding query filter params (name=image) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:35.198404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:35.201189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:35.216788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 528/1049] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:35.218033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:35.269327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:35.354619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b097c-4123-43fa-be41-22410d1b4a1d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:35.419982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:35.420618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:35.420889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:35.421144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:35.421796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] RBAC: Adding query filter params (type=image) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:35.422088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:35.423519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228c7c00-4700-43ae-bb9f-b90acbca748e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:35.435125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 522/1050] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:35.442883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:35.464003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:35.606368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c0177c-2c7e-4350-810d-1822eabfcdb7 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:35.650350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:35.650972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:35.651376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:35.651664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] PATH_INFO: `/v3/endpoints` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:35.652343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:35.653751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010c66f-e0e3-42cb-8833-9f5f1286e7db None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:35.688255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 529/1051] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:23:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:37.045633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f76f88-205a-4553-9699-6312cf8e7f4c None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.046269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f76f88-205a-4553-9699-6312cf8e7f4c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.046269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f76f88-205a-4553-9699-6312cf8e7f4c None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.047822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 523/1052] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:37.055008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562142a3-7115-422a-951c-c0e6f0c1ff5d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.055651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562142a3-7115-422a-951c-c0e6f0c1ff5d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.055651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562142a3-7115-422a-951c-c0e6f0c1ff5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.078211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-562142a3-7115-422a-951c-c0e6f0c1ff5d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:37.080370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-562142a3-7115-422a-951c-c0e6f0c1ff5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:37.198474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:37.219452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 530/1053] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:37.259396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f76f88-205a-4553-9699-6312cf8e7f4c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.271389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963f7c-44d7-44c7-9f08-449aa9d347a4 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.271696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963f7c-44d7-44c7-9f08-449aa9d347a4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.271943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52963f7c-44d7-44c7-9f08-449aa9d347a4 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.273452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 531/1054] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:23:37.278617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.278895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.279193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.300994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:37.302192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:37.387436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 532/1055] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:37.407469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f76f88-205a-4553-9699-6312cf8e7f4c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:37.411985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:37.414265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.463776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:37.464507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.464842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.465133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.480220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:37.485475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:37.487113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:37.488562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:37.491339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 524/1056] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:37.494602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.593168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812c1545-89e7-45d3-b95b-e6f329a09e0a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:37.629416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:37.637413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf24fa-da75-41b3-bf47-ec8aea56e7c7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:37.640559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:37.642361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.642875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.643390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.656763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:37.663197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:37.664250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:37.665151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:37.672289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 533/1057] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:37.673429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.680388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:37.681016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.681293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.681595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.682271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:37.682573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:37.683994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:37.695065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 525/1058] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:37.703778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.750431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:37.826633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368f597-f1fd-4a1c-8683-5af055f694d6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:37.833853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34def4-ee10-4205-a496-554c8ae8915c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:37.876931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:37.877580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.877767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.878102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.878657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:37.878998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:37.880600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:37.890773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:37.891795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 534/1059] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:37.891974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:37.892386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:37.892810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] PATH_INFO: `/v3/users/94e6f42567444340a7db03ead52f1235` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:37.898253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.911401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] RBAC: Authorizing `identity:get_user(user_id=94e6f42567444340a7db03ead52f1235)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:37.913277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:37.915255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 526/1060] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/users/94e6f42567444340a7db03ead52f1235 => generated 285 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:37.921129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:37.958016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.017576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42b9dca-cfc7-4aaf-8a71-bbf9b7a21963 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.018367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.035545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69c7e66-907e-46a3-828c-c70d564f3c21 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.053321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.053968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.054242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.054509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.066151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.068221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.070705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.071362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.071616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.071888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-6c91a019-2286-42d4-a35a-5c378d208dd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:38.084030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:38.085607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:38.086597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:38.086597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:38.086597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:38.086597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 535/1061] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:38.086597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc734-5be7-44fd-9791-25a311926407 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.086922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc734-5be7-44fd-9791-25a311926407 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.086922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ecc734-5be7-44fd-9791-25a311926407 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.088408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 536/1062] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:38.091351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc734-5be7-44fd-9791-25a311926407 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.156094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.157802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.200359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.206525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ecc734-5be7-44fd-9791-25a311926407 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.259401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.260343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.260797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.261235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.262268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:38.262484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.263160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.264754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8b4c16-2a32-40ea-bdeb-261a0832f5a3 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.282982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 537/1063] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:38.282982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.282982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.282982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.286256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 527/1064] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:38.287439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.300051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:38.301291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:38.379159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 538/1065] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:38.379159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.412107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef97a49-5658-43fc-9248-86fa5da1e370 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.447311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.447950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.448228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.448500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.456523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.457997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:38.470235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:38.471346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 528/1066] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:38.472327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.496398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.508784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d2ace3-747f-44c9-bf82-6c233376340b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.535532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.548968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.549614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.549897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.550173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.551363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.552851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.554524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 539/1067] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:38.555599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.604087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f092385a-b680-4234-80ed-02a17977b245 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.639820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.640437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.640713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.640979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.693592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc73b0-e1e7-4367-912b-9051736550c9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.714346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.715869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.745512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.746138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.746403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.746669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.747454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:38.747755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.749208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.768981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 540/1068] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:38.770221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.779788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.790417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:38.850068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 529/1069] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:38.852087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.896511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b0d6e-b5cb-4a16-81ed-5e132de0de02 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:38.936995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:38.937613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:38.937893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:38.938167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:38.938907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=1ba84b20a1054b598d46b272961c9397, user.id=94e6f42567444340a7db03ead52f1235) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:38.939262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:38.940761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:38.946881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 541/1070] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/role_assignments?user.id=94e6f42567444340a7db03ead52f1235&scope.project.id=1ba84b20a1054b598d46b272961c9397&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:38.947674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:38.972527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e0826-e255-48f6-9b53-381b1953dcb9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.033645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.034800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.034800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.035173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.048002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.049921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.050908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:39.054853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:39.074386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:39.076350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:39.077265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:39.077265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:39.077265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:39.077265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 530/1071] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:39.078127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341961e-7d2e-4831-b8f7-93a87b7887aa None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.081963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:39.124396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.124396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.124396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.124762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.138159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.139736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:39.156098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:39.157145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:39.158071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:39.158071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:39.158071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:39.158071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 542/1072] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:39.163611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:39.209655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc92beb-c8ed-41e8-9f7a-1c852c37c5e6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.263835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.264527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.264852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.265086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.265761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:39.266060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.268205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.282733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 531/1073] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:39.293393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:39.305309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca1d4-ac0a-479b-b7ce-a06c1d275129 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.350064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:39.358540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.359529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.359986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.360429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.361484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:39.361974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.364085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.380246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:39.390342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 543/1074] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:39.469477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f15403-53e1-4099-bb56-bb256eecaebe None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.537146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.538682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.539445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.540259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.553897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.557164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:39.578993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:39.580122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:39.581055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:39.587628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 532/1075] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:39.590745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:39.734887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaefe73d-68bf-497b-9e5e-d9f3be5d1c0d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:39.797173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:39.798194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:39.798661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:39.799170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:39.800466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:39.801006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:39.803535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e776f7c6-65d4-4d4e-8fee-6b5b89dcd11a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:39.825055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 544/1076] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:39.831579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:39.990455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e8942-8774-4ccb-b31e-244c5170408e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:40.012201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:40.041112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:40.042230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:40.042706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:40.043208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/f7a2a9d4dbb24ce4bf02ee811622cf3b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:40.045971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=8e18ece85f014104b3c8fe92355b18a9, role_id=f7a2a9d4dbb24ce4bf02ee811622cf3b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:40.047710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d3955-c6a6-45f2-8d5e-b22fc4edff8e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:40.063750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 533/1077] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:23:39 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/8e18ece85f014104b3c8fe92355b18a9/roles/f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:23:41.002793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b405-269a-4da6-8d0b-1cac50128809 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.003478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b405-269a-4da6-8d0b-1cac50128809 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.003478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67b405-269a-4da6-8d0b-1cac50128809 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.004729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 545/1078] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:41.011124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.011633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.011794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.030832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:41.032079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:41.101597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 534/1079] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:41.124819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67b405-269a-4da6-8d0b-1cac50128809 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:41.184475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:41.241577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f67b405-269a-4da6-8d0b-1cac50128809 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:41.278519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:41.279097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.279390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.279662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.291006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:41.295707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:41.296716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:41.297770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 546/1080] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:23:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:41.303568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:41.413990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56958894-eeaf-443c-a5f2-c19e273bd713 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:41.422242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:41.454663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:41.455310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.455588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.455855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.456557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:41.456859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:41.458318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:41.467315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 535/1081] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:23:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:41.473320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:41.580168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a4d03-f146-4ba8-99da-d641bba404ee None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:41.594594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:41.620272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:41.620918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.621247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.621630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] PATH_INFO: `/v3/users/94e6f42567444340a7db03ead52f1235` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.622955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] RBAC: Authorizing `identity:get_user(user_id=94e6f42567444340a7db03ead52f1235)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:41.624649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:41.626515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 547/1082] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:41 2022] GET /identity/v3/users/94e6f42567444340a7db03ead52f1235 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:41.633222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:41.742651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945959b-80fa-453e-9304-a8e48488eba4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:41.779718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:41.780362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.780637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.780901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.782077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:41.783588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c4cc7c-7342-466e-b852-05976a1c1866 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:41.785286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 536/1083] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:41 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:41.791706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:41.903808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c792edc-0e17-44db-8006-69b7b73dbbca None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:41.941747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:41.942673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:41.943027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:41.943548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397/users/94e6f42567444340a7db03ead52f1235/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:41.945996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba84b20a1054b598d46b272961c9397, user_id=94e6f42567444340a7db03ead52f1235, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:41.948194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebbfee-c4c4-4657-be94-9e98ad42a659 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:41.964709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 548/1084] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:23:41 2022] PUT /identity/v3/projects/1ba84b20a1054b598d46b272961c9397/users/94e6f42567444340a7db03ead52f1235/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:23:42.103537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5f3f4-fa71-4619-9756-e1fd3a69c20e None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:42.104201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5f3f4-fa71-4619-9756-e1fd3a69c20e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:42.104201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c5f3f4-fa71-4619-9756-e1fd3a69c20e None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:42.105775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 537/1085] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:42.112356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:42.113198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:42.113198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:42.135235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:42.136486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:42.184078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:42.225005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 549/1086] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:42.253126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5f3f4-fa71-4619-9756-e1fd3a69c20e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:42.298211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:42.371953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c5f3f4-fa71-4619-9756-e1fd3a69c20e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:42.416283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:42.416936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:42.417216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:42.417487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:42.430193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:42.435447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:42.437414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:42.438898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 538/1087] 10.222.0.33 () {56 vars in 1120 bytes} [Mon Aug 8 13:23:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:42.444309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:42.553835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f374e-cc00-494f-a11b-614ace80cb95 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:42.598631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:42.599322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:42.599600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:42.599877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:42.600562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:42.600866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:42.602271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:42.611760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 550/1088] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:23:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:42.618409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:42.743675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:42.751810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36320e8-c888-445f-ad39-4f9294ff3df0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:42.793635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:42.794303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:42.794582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:42.794858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:42.809660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:42.811205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:42.832601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:42.833742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:42.834646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:42.834646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:42.834646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:42.840497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 539/1089] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:42.841786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:42.878744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:42.954879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f08f2-0cd2-4a70-885b-ec8b7e86218f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:43.017701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:43.018645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:43.019048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:43.019523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:43.020746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:43.021269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:43.023936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:43.051015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 551/1090] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:43.053164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:43.177037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fbd59-a426-4630-9d60-79257c6a614e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:43.217097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:43.217779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:43.218062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:43.218333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:43.227943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:43.229413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:23:43.242207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:43.250178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:43.252143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 540/1091] 10.222.0.33 () {56 vars in 1093 bytes} [Mon Aug 8 13:23:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:43.254875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:43.374360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:43.389321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bef18-cac0-4303-b40c-4247c0d66f95 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:43.426668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:43.427426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:43.427699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:43.427965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:43.428706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:43.429007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:43.430452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:43.449022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 552/1092] 10.222.0.33 () {56 vars in 1133 bytes} [Mon Aug 8 13:23:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:43.455001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:43.467267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:43.559588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ff9e7-2c9f-46b8-9ec8-efce47f65765 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:43.595767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b400669-f862-4863-ac61-5886899df152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:43.596391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400669-f862-4863-ac61-5886899df152 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:43.596665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400669-f862-4863-ac61-5886899df152 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:43.596935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b400669-f862-4863-ac61-5886899df152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:43.667358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b400669-f862-4863-ac61-5886899df152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:43.668839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b400669-f862-4863-ac61-5886899df152 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:43.681645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:43.790565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 541/1093] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:23:43.796895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:43.912545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec480a2-818a-45ad-b2d5-a3642d34a823 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:43.948105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:43.948629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:43.948910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:43.949179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:43.959926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:43.961398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-aa0c3478-81f6-475c-b16d-f38af327ce1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:23:43.976597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:43.978467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:23:43.980345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:23:43.980345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:23:43.980345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:43.982761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 553/1094] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:23:43.983927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b400669-f862-4863-ac61-5886899df152 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.036447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfa9c-4179-44a0-a55b-7a283bca3b41 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.036886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfa9c-4179-44a0-a55b-7a283bca3b41 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.037327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031cfa9c-4179-44a0-a55b-7a283bca3b41 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.039607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 554/1095] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:44.046941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.047294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.047708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.055637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.070109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:44.071637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:44.092633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b400669-f862-4863-ac61-5886899df152 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.102352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.134529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.135188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.135471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.135744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.136399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:44.136697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.138100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.154651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 542/1096] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:44.160489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.162453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 555/1097] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:23:44.188143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.277117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bc241-30eb-413b-b0aa-dba65e49d5d2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.299829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0e1f5-77ee-40fc-b5f5-6184e3040771 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.313608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.314266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.314538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.314801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.322789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.324323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.332340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:23:44.336313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:44.337847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:23:44.339685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 543/1098] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:44.340652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.340652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.340921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.341267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.343703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.353661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:23:44.358501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:23:44.359593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:23:44.360431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 556/1099] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:23:44.366227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.376934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.492250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae497d-7823-4ab4-af59-9ed74ceddc8b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.508417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a507410-6cfe-465e-9187-5a2867caec25 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.530574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.531288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.531586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.531925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.532799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:44.533265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.534982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.544681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.545301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.545577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.545849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.546523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:44.546820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.548241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.550026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 544/1100] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:44.556342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.568142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 557/1101] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:44.569906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.629556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.659197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:44.674234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc277c-e69f-4fd7-98e0-665a2c04162d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.681270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4e498e-cf2b-4d51-a121-744dee574b77 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.716742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.717721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.718167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.718612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.719935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] RBAC: Adding query filter params (role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=8e18ece85f014104b3c8fe92355b18a9) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:44.720490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.723123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.724464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.725467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.725895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.726329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] PATH_INFO: `/v3/users/94e6f42567444340a7db03ead52f1235` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.727905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] RBAC: Authorizing `identity:get_user(user_id=94e6f42567444340a7db03ead52f1235)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:44.729724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:44.731328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 545/1102] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/role_assignments?user.id=8e18ece85f014104b3c8fe92355b18a9&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=f7a2a9d4dbb24ce4bf02ee811622cf3b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:44.731514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 558/1103] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/users/94e6f42567444340a7db03ead52f1235 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:44.737680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:44.874414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba69cd6d-acfe-4ffd-8f22-889330eadef4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:44.914879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:44.915537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:44.915818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:44.916091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:44.941910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:45.022187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:45.023519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:45.148732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 546/1104] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:45.155013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:45.273397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e5d2c6-bd0d-44af-93fb-c2f4689b196b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:45.310123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:45.324444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:45.325108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:45.325385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:45.325665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:45.326839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:45.328353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894845e8-1ae2-4c55-af19-851895c94086 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:45.330054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 559/1105] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:45 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:45.336576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:45.401034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:45.467274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d937a-8f72-431e-a5f9-5cb4c5286e9b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:45.512583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:45.513213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:45.513490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:45.513763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:45.514499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=1ba84b20a1054b598d46b272961c9397, user.id=94e6f42567444340a7db03ead52f1235) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:45.514802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:45.516333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cfc3c9-337f-43eb-a741-fac1ea204b11 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:45.522987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 547/1106] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:45 2022] GET /identity/v3/role_assignments?user.id=94e6f42567444340a7db03ead52f1235&scope.project.id=1ba84b20a1054b598d46b272961c9397&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:47.559467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bba33-7692-44a1-90f4-ac1caeb2eb20 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:47.560096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bba33-7692-44a1-90f4-ac1caeb2eb20 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:47.560096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bba33-7692-44a1-90f4-ac1caeb2eb20 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:47.561346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 560/1107] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:47.568212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:47.568748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:47.568748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:47.588400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:47.589616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:47.660399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 548/1108] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:47.684788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bba33-7692-44a1-90f4-ac1caeb2eb20 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:47.800429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bba33-7692-44a1-90f4-ac1caeb2eb20 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:47.835883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:47.836532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:47.836786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:47.837076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:47.838336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:47.839868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ac3aa-8559-4562-9fac-5871c2307c0d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:47.841617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 561/1109] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:47.848151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:47.911145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:47.973870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc8179-1069-40c5-9999-9d43de8bd348 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:48.033330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:48.034330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:48.034813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:48.035350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:48.036719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:48.039267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7056f0e3-0111-4f76-8b3a-3c95c52e4d9f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:48.072320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 549/1110] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:23:47 2022] POST /identity/v3/users => generated 286 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:23:50.138077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25289-0732-4b26-a62b-01b80adaad3a None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:50.138835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25289-0732-4b26-a62b-01b80adaad3a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:50.138835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e25289-0732-4b26-a62b-01b80adaad3a None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:50.139989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 562/1111] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:50.145724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:50.146257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:50.146355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:50.165378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:50.166609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:50.238802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 550/1112] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:50.263110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25289-0732-4b26-a62b-01b80adaad3a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:50.333817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:50.390145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e25289-0732-4b26-a62b-01b80adaad3a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:50.433295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:50.434402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:50.434877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:50.435348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] PATH_INFO: `/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:50.437829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] RBAC: Authorizing `identity:get_role(role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:50.440274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:50.442875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 563/1113] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:50 2022] GET /identity/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 223 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:50.449220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:50.480838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:50.566712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc84a1a-8b9c-4fcc-a24f-f490f233cd51 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:50.602736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:50.603439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:50.603704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:50.603971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] PATH_INFO: `/v3/users/d702455d0ceb4827903304bc682d2e05` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:50.618064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] RBAC: Authorizing `identity:get_user(user_id=d702455d0ceb4827903304bc682d2e05)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:50.619576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:50.621281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 551/1114] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:50 2022] GET /identity/v3/users/d702455d0ceb4827903304bc682d2e05 => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:50.627391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:50.739960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39cfe0-dd10-45a2-bb39-e692f870adb1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:50.769366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:50.784340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:50.785392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:50.785872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:50.786370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:50.890890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:50.892401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:51.001911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 564/1115] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:23:51.008679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:51.113510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:51.124396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6267b27f-bdaf-49c1-bdbb-5ad719bb1b75 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:51.160732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:51.161531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:51.161779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:51.162051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:51.163229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:51.164675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83e7399-dd69-4143-afcc-7adf8f564f27 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:51.166357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 552/1116] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:51 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:51.172918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:51.200230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:51.285131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae158a6-f0dd-4652-a4e0-690ea56a904a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:51.329632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:51.330260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:51.330532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:51.330801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:51.331601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=1ba84b20a1054b598d46b272961c9397, user.id=d702455d0ceb4827903304bc682d2e05) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:51.331893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:51.333437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a776944d-ed1a-4e54-8d2f-6681c3f90c6c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:51.340103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 565/1117] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:51 2022] GET /identity/v3/role_assignments?user.id=d702455d0ceb4827903304bc682d2e05&scope.project.id=1ba84b20a1054b598d46b272961c9397&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:53.334236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8e47f-c416-4c13-b821-1d49109f91c8 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:53.334823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8e47f-c416-4c13-b821-1d49109f91c8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:53.334823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8e47f-c416-4c13-b821-1d49109f91c8 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:53.336207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 553/1118] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:53.342642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:53.343192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:53.343192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:53.361738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:53.362937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:53.432044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 566/1119] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:53.454293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8e47f-c416-4c13-b821-1d49109f91c8 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:53.560798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8e47f-c416-4c13-b821-1d49109f91c8 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:53.596319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:53.596873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:53.597151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:53.597420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] PATH_INFO: `/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:53.598939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] RBAC: Authorizing `identity:get_role(role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:53.600385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:53.601737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 554/1120] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:53 2022] GET /identity/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 223 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:53.607941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:53.629575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:53.713696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4ef4e-d9ad-4f55-936e-71c076bbd0af None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:53.751569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:53.752206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:53.752482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:53.752752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] PATH_INFO: `/v3/users/d702455d0ceb4827903304bc682d2e05` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:53.753915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] RBAC: Authorizing `identity:get_user(user_id=d702455d0ceb4827903304bc682d2e05)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:53.755420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:53.757390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 567/1121] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:53 2022] GET /identity/v3/users/d702455d0ceb4827903304bc682d2e05 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:53.763489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:53.791747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:53.870409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54cfdc1-b3e5-416b-8c5f-236b313b1fdb None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:53.905544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:53.906176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:53.906448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:53.906715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:53.907850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:53.909284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9b58ee-027d-43af-a03e-690b802988db None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:53.910909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 555/1122] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:53 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:53.917979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:54.011378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:54.024856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce82942-51ce-4ca3-8a36-70ad223b6b88 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:54.061658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:54.062326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:54.062605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:54.062876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397/users/d702455d0ceb4827903304bc682d2e05/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:54.065282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] RBAC: Authorizing `identity:create_grant(project_id=1ba84b20a1054b598d46b272961c9397, user_id=d702455d0ceb4827903304bc682d2e05, role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:54.066742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594148ef-d365-43a4-81ad-faa3dcb3e09b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:54.081281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 568/1123] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:23:53 2022] PUT /identity/v3/projects/1ba84b20a1054b598d46b272961c9397/users/d702455d0ceb4827903304bc682d2e05/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:23:56.053968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac1f3-dda0-459f-aa12-f324109518d8 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.054546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac1f3-dda0-459f-aa12-f324109518d8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.054546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfac1f3-dda0-459f-aa12-f324109518d8 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.055889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 556/1124] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:56.061612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.062147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.062147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.082929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:56.084201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:56.155164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 569/1125] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:56.178589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac1f3-dda0-459f-aa12-f324109518d8 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:56.275143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:56.283572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfac1f3-dda0-459f-aa12-f324109518d8 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:56.319363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:56.319903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.320158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.320427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] PATH_INFO: `/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.321824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] RBAC: Authorizing `identity:get_role(role_id=6a6dcf01fe1c4dbda05b54fa0f59199a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:56.323236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:56.324573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 557/1126] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:56 2022] GET /identity/v3/roles/6a6dcf01fe1c4dbda05b54fa0f59199a => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:56.330704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:56.412332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:56.435890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc05a05-a802-4326-8c38-d91b0a145ea6 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:56.473622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:56.474271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.474552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.474829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] PATH_INFO: `/v3/users/d702455d0ceb4827903304bc682d2e05` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.476111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] RBAC: Authorizing `identity:get_user(user_id=d702455d0ceb4827903304bc682d2e05)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:56.477598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:56.479389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 570/1127] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:23:56 2022] GET /identity/v3/users/d702455d0ceb4827903304bc682d2e05 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:56.485759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:56.599558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e27c83-eca2-4823-ac1f-626b3f1df502 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:56.636549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:56.637169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.637447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.637721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.668470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:56.708605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:56.710087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:56.821966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 558/1128] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:23:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:23:56.828314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:56.946637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fe898-05d8-484b-9c66-d3a26354e184 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:56.982023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:56.982655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:56.982926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:56.983229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] PATH_INFO: `/v3/projects/1ba84b20a1054b598d46b272961c9397` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:56.984402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] RBAC: Authorizing `identity:get_project(project_id=1ba84b20a1054b598d46b272961c9397)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:56.985854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1fe51-7bae-40e7-bd07-f0ad7d711d90 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:56.987586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 571/1129] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:23:56 2022] GET /identity/v3/projects/1ba84b20a1054b598d46b272961c9397 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:23:56.994167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:57.064861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:57.110538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fb0c48-5c57-4a67-b443-86b9ebc05d66 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:57.156443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:57.157075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:57.157353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:57.157623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:57.158372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] RBAC: Adding query filter params (role.id=6a6dcf01fe1c4dbda05b54fa0f59199a, scope.project.id=1ba84b20a1054b598d46b272961c9397, user.id=d702455d0ceb4827903304bc682d2e05) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:23:57.158680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:57.160204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2142ad-5ae2-4930-bf07-0e80b94d635e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:57.167030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 559/1130] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:23:56 2022] GET /identity/v3/role_assignments?user.id=d702455d0ceb4827903304bc682d2e05&scope.project.id=1ba84b20a1054b598d46b272961c9397&role.id=6a6dcf01fe1c4dbda05b54fa0f59199a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:59.191853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0f4bb-4536-4e76-b7c2-38459c88f847 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:59.192426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0f4bb-4536-4e76-b7c2-38459c88f847 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:59.192426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad0f4bb-4536-4e76-b7c2-38459c88f847 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:59.193748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 572/1131] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:23:59.199242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:59.199667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:59.199778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:59.219707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:23:59.220924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:23:59.291758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 560/1132] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:23:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:23:59.315237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0f4bb-4536-4e76-b7c2-38459c88f847 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:59.332134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:59.424939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad0f4bb-4536-4e76-b7c2-38459c88f847 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:59.460933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:59.461486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:59.461762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:59.462026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:59.463205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:59.464642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66035e9-cd02-4f7e-b89f-5463bf059a67 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:59.466223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 573/1133] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:23:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:23:59.472460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:23:59.533812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:23:59.580055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24707fcb-1821-4420-b1c5-4f750a3fee4b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:23:59.619421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:23:59.620185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:23:59.620460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:23:59.620840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:23:59.621555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:23:59.623002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591d95-3b1d-489a-8494-5a0fb46f871e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:23:59.653393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 561/1134] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:23:59 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:24:01.663396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26baf2-5a15-4374-9abb-ceacda5b18fa None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:01.663972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26baf2-5a15-4374-9abb-ceacda5b18fa None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:01.663972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26baf2-5a15-4374-9abb-ceacda5b18fa None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:01.665291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 574/1135] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:01.670732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:01.671177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:01.671309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:01.690055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:01.691291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:01.762181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 562/1136] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:01.786474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26baf2-5a15-4374-9abb-ceacda5b18fa None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:01.874142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:01.894214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26baf2-5a15-4374-9abb-ceacda5b18fa None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:01.929944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:01.930493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:01.930772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:01.931047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:01.932262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:01.933732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb5617-a52e-4362-8164-1e04e1f10482 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:01.935347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 575/1137] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:24:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:01.941354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:01.981627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:02.043814 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29908d4d-a8c8-42a1-ac2e-76930208a6a9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:02.078692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:02.079351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:02.079629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:02.079904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:02.080545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:02.081985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79faa01b-2288-460e-8a8b-a8c589f9d799 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:02.105759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 563/1138] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:24:01 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:24:04.124280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ea38-8c89-4e28-91fe-a3ae953377cb None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.124871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ea38-8c89-4e28-91fe-a3ae953377cb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.124871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ea38-8c89-4e28-91fe-a3ae953377cb None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.126177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 576/1139] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:04.132014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.132436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.132563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.151377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:04.152584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:04.222483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 564/1140] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:04.245658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ea38-8c89-4e28-91fe-a3ae953377cb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:04.351855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ea38-8c89-4e28-91fe-a3ae953377cb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:04.388126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:04.388776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.389048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.389322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.400468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:04.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:04.406502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:04.407624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 577/1141] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:04.412960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:04.437993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:04.514202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e22140-8621-4989-bffd-1b088cf40d92 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:04.549552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:04.550161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.550437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.550707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.551495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:04.551792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:04.553180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:04.561331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 565/1142] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:24:04.567593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:04.582321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:04.673357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eddf7d2-a9e1-44a6-8de3-3e70be8f0f99 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:04.708242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:04.708871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.709151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.709422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] PATH_INFO: `/v3/users/1c117644d9734b508369fc5fa7cedd86` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.723347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] RBAC: Authorizing `identity:get_user(user_id=1c117644d9734b508369fc5fa7cedd86)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:04.724809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:04.726458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 578/1143] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:04 2022] GET /identity/v3/users/1c117644d9734b508369fc5fa7cedd86 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:04.732547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:04.820225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:04.837050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec6d5ae-392f-418c-832f-45742e0058f0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:04.872991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:04.873643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:04.873922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:04.874190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:04.950039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:04.951429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:05.034765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:05.050693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 566/1144] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:24:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:24:05.057704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:05.132947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:05.169453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593c9558-4cf9-4a7a-8516-e6df71b5736a None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:05.208054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:05.208593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:05.208865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:05.209128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] PATH_INFO: `/v3/projects/366c6866320244bcbb0257be381ee4c5` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:05.210288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] RBAC: Authorizing `identity:get_project(project_id=366c6866320244bcbb0257be381ee4c5)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:05.211764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee594ad-96ec-4f80-b91c-0e25f3f8220e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:05.213427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 579/1145] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:05 2022] GET /identity/v3/projects/366c6866320244bcbb0257be381ee4c5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:05.220438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:05.329445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54afc9-fb78-4f12-82ab-c83f6f6749c3 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:05.364565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:05.365203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:05.365483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:05.365748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:05.366472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=366c6866320244bcbb0257be381ee4c5, user.id=1c117644d9734b508369fc5fa7cedd86) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:05.366774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:05.368236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dbda2-576d-4bc2-8685-5221d9e28172 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:05.374241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 567/1146] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:24:05 2022] GET /identity/v3/role_assignments?user.id=1c117644d9734b508369fc5fa7cedd86&scope.project.id=366c6866320244bcbb0257be381ee4c5&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:07.429908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0de1-c01d-48c5-a7de-6d0cec74df1f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:07.430455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0de1-c01d-48c5-a7de-6d0cec74df1f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:07.430455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa0de1-c01d-48c5-a7de-6d0cec74df1f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:07.431808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 580/1147] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:07.437426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:07.437834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:07.437974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:07.457196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:07.458420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:07.507102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:07.530391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 568/1148] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:07.553194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa0de1-c01d-48c5-a7de-6d0cec74df1f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:07.668001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa0de1-c01d-48c5-a7de-6d0cec74df1f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:07.706538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:07.707229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:07.707504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:07.707772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:07.710596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:07.719276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:07.724043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:07.725159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:07.726212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 581/1149] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:07.731474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:07.832569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1720fb4-a3a7-46e1-8be1-72ca460b36f9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:07.866847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:07.867488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:07.867762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:07.868034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:07.868705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:07.868981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:07.870367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:07.878578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 569/1150] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:07.885193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:07.997058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad7053-91c2-458d-b8ba-c1e559572fe9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:08.035434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:08.036071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:08.036355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:08.036611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] PATH_INFO: `/v3/users/1c117644d9734b508369fc5fa7cedd86` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:08.037869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] RBAC: Authorizing `identity:get_user(user_id=1c117644d9734b508369fc5fa7cedd86)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:08.039357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:08.040957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 582/1151] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:07 2022] GET /identity/v3/users/1c117644d9734b508369fc5fa7cedd86 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:08.049774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:08.099555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:08.158864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafbc39a-ecb9-49f0-a594-88efd419b6e7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:08.193994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:08.194619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:08.194892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:08.195160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] PATH_INFO: `/v3/projects/366c6866320244bcbb0257be381ee4c5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:08.196653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] RBAC: Authorizing `identity:get_project(project_id=366c6866320244bcbb0257be381ee4c5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:08.198314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fb49d-0802-4375-84cf-536ce85d60d8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:08.200185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 570/1152] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:08 2022] GET /identity/v3/projects/366c6866320244bcbb0257be381ee4c5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:08.207261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:08.274069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:08.315721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f9475-72a3-488b-8266-179aa5d7a7fe None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:08.352637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:08.353371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:08.353663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:08.353959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] PATH_INFO: `/v3/projects/366c6866320244bcbb0257be381ee4c5/users/1c117644d9734b508369fc5fa7cedd86/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:08.356098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] RBAC: Authorizing `identity:create_grant(project_id=366c6866320244bcbb0257be381ee4c5, user_id=1c117644d9734b508369fc5fa7cedd86, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:08.357662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0cb504-a23e-46a2-9887-e28e8526ef5e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:08.372397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 583/1153] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:24:08 2022] PUT /identity/v3/projects/366c6866320244bcbb0257be381ee4c5/users/1c117644d9734b508369fc5fa7cedd86/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:24:10.378381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f473-04bf-406e-9f46-0726bf4e1cf4 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:10.378941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f473-04bf-406e-9f46-0726bf4e1cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:10.378941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e196f473-04bf-406e-9f46-0726bf4e1cf4 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:10.381202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 571/1154] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:10.386863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:10.387359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:10.387452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:10.406491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:10.407715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:10.484787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 584/1155] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:24:10.508880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196f473-04bf-406e-9f46-0726bf4e1cf4 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:10.614800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e196f473-04bf-406e-9f46-0726bf4e1cf4 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:10.625772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:10.651988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:10.652635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:10.652912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:10.653190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:10.663975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:10.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:10.669823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:10.670863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:24:10.674982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 572/1156] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:10.676149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:10.742636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:10.793283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9683653-6147-43ed-a0fc-f9d57509f7aa None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:10.829083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:10.829689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:10.829965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:10.830232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:10.830894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:10.831220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:10.832668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:10.842220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 585/1157] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:24:10.849139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:10.963494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695c365-2376-4e9f-bbab-e7ee4a7fa595 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:11.000166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:11.000819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:11.001096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:11.001360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] PATH_INFO: `/v3/users/1c117644d9734b508369fc5fa7cedd86` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:11.002793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] RBAC: Authorizing `identity:get_user(user_id=1c117644d9734b508369fc5fa7cedd86)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:11.004326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:11.006317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 573/1158] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:10 2022] GET /identity/v3/users/1c117644d9734b508369fc5fa7cedd86 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:11.012642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:11.117854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec583d70-1ebd-4ece-a75d-66d90747fc87 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:11.135457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:11.154197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:11.154810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:11.155105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:11.155392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:11.223300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:11.224670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:11.325086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 586/1159] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:24:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:24:11.331381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:11.370945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:11.432289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56b1dc-97aa-4eb2-9e1f-ba5bed335933 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:11.466843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:11.467515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:11.467793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:11.468060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] PATH_INFO: `/v3/projects/366c6866320244bcbb0257be381ee4c5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:11.469177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] RBAC: Authorizing `identity:get_project(project_id=366c6866320244bcbb0257be381ee4c5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:11.470621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f332e-87cd-4059-896b-42aeaca1e546 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:11.472296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 574/1160] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:11 2022] GET /identity/v3/projects/366c6866320244bcbb0257be381ee4c5 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:11.478889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:11.496760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:11.580085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e8bd1-3158-4291-a186-3ae5622bcef8 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:11.615694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:11.616300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:11.616573 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:11.616851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:11.617567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=366c6866320244bcbb0257be381ee4c5, user.id=1c117644d9734b508369fc5fa7cedd86) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:11.617872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:11.619346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a11969-1f48-4995-ae8c-5e4b5a9333b5 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:11.625338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 587/1161] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:24:11 2022] GET /identity/v3/role_assignments?user.id=1c117644d9734b508369fc5fa7cedd86&scope.project.id=366c6866320244bcbb0257be381ee4c5&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:13.622116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f66d-aed9-4f59-8f4d-76a4a9cf88e9 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:13.622706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f66d-aed9-4f59-8f4d-76a4a9cf88e9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:13.622706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06f66d-aed9-4f59-8f4d-76a4a9cf88e9 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:13.624028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 575/1162] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:13.630129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:13.630960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:13.631053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:13.650905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:13.652345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:13.726268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 588/1163] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:13.748686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06f66d-aed9-4f59-8f4d-76a4a9cf88e9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:13.860261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06f66d-aed9-4f59-8f4d-76a4a9cf88e9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:13.869697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:13.899059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:13.899704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:13.899981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:13.900255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:13.973013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:13.974394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:14.073660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 576/1164] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:24:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:24:14.080703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:14.093687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:14.195495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f6ede-d10a-4633-a216-8496d6580403 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:14.234954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:14.235530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:14.235807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:14.236081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:14.244127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:14.245625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c6536b61-e30a-479d-bc3f-14289661a03d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:14.258842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:14.260194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:14.261096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 589/1165] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:24:14 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:14.266640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:14.275742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:14.377783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0960-0fcd-4c5e-945e-ed4b84fcedea None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:14.413868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:14.414480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:14.414761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:14.415034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:14.415719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:14.416005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:14.417381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21739210-73f4-4bda-bc39-54a430a31d0e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:14.424761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 577/1166] 10.222.0.33 () {56 vars in 1115 bytes} [Mon Aug 8 13:24:14 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:16.388478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3a5b-e47f-4ea0-91b0-13f9bde967ec None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:16.388930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3a5b-e47f-4ea0-91b0-13f9bde967ec None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:16.389052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e3a5b-e47f-4ea0-91b0-13f9bde967ec None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:16.390364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 590/1167] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:16.396714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935397e7-efdc-445e-8b2a-365d9992068d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:16.397140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935397e7-efdc-445e-8b2a-365d9992068d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:16.397261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935397e7-efdc-445e-8b2a-365d9992068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:16.415301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-935397e7-efdc-445e-8b2a-365d9992068d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:16.416512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-935397e7-efdc-445e-8b2a-365d9992068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:16.484316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:16.489778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 578/1168] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:24:16.516890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3a5b-e47f-4ea0-91b0-13f9bde967ec None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:16.600444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:16.634580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e3a5b-e47f-4ea0-91b0-13f9bde967ec None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:16.669037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:16.669638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:16.669909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:16.670175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:16.670844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:16.672254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb6b34-2678-4cd5-b34e-67bade8ba634 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:16.690688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 591/1169] 10.222.0.33 () {60 vars in 1138 bytes} [Mon Aug 8 13:24:16 2022] POST /identity/v3/domains => generated 267 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:24:18.705902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12019a3-315a-4f0b-842d-b0b994af84cb None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:18.706528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12019a3-315a-4f0b-842d-b0b994af84cb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:18.706528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12019a3-315a-4f0b-842d-b0b994af84cb None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:18.707907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 579/1170] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:18.713392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:18.713797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:18.713937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:18.732755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:18.733960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:18.805539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 592/1171] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:18.828538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12019a3-315a-4f0b-842d-b0b994af84cb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:18.932105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12019a3-315a-4f0b-842d-b0b994af84cb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:18.967187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:18.967749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:18.968024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:18.968297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] PATH_INFO: `/v3/domains/8365ed5388ff41898de5ebb33828a1e6` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:18.979185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8365ed5388ff41898de5ebb33828a1e6)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:18.980645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e694407-ab47-429a-ab62-ac469265d525 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:18.982271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 580/1172] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Aug 8 13:24:18 2022] GET /identity/v3/domains/8365ed5388ff41898de5ebb33828a1e6 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:18.988682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:19.052021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:19.093463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5313950-707b-4d13-ae6c-dd24f10cc800 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:19.128098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:19.128757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:19.129031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:19.129282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:19.129923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:19.131755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8a9a1c-edf8-4fd7-9aee-958899fd938f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:19.160002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 593/1173] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:24:18 2022] POST /identity/v3/projects => generated 361 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:24:21.164439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7c5e-bdda-4e89-bcd8-7e1216d5bc4a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:21.165081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7c5e-bdda-4e89-bcd8-7e1216d5bc4a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:21.165081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7c5e-bdda-4e89-bcd8-7e1216d5bc4a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:21.166334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 581/1174] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:21.171769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:21.172218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:21.172324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:21.190689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:21.191944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:21.262261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 594/1175] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:21.286745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce7c5e-bdda-4e89-bcd8-7e1216d5bc4a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:21.351388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:21.395503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce7c5e-bdda-4e89-bcd8-7e1216d5bc4a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:21.432245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:21.432779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:21.433027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:21.433322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] PATH_INFO: `/v3/domains/8365ed5388ff41898de5ebb33828a1e6` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:21.434467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] RBAC: Authorizing `identity:get_domain(domain_id=8365ed5388ff41898de5ebb33828a1e6)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:21.435941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d8c6a-ff23-49c5-9e00-746cc180ecda None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:21.437519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 582/1176] 10.222.0.33 () {56 vars in 1183 bytes} [Mon Aug 8 13:24:21 2022] GET /identity/v3/domains/8365ed5388ff41898de5ebb33828a1e6 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:21.443931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:21.486939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:21.545149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952b597-f3ee-440f-a378-e5c7fb1698b8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:21.579804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:21.580404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:21.580682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:21.580936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:21.581681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:21.583152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f2f35-1ef3-44c2-af4a-2cd817581e2f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:21.605903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 595/1177] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:24:21 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:24:23.629217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9592-c3cf-42e9-a6ca-86a3a060ef78 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:23.629801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9592-c3cf-42e9-a6ca-86a3a060ef78 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:23.629801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca9592-c3cf-42e9-a6ca-86a3a060ef78 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:23.631473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 583/1178] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:23.636925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:23.637373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:23.637373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:23.655684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:23.656849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:23.725565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 596/1179] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:24:23.748981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca9592-c3cf-42e9-a6ca-86a3a060ef78 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:23.855014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca9592-c3cf-42e9-a6ca-86a3a060ef78 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:23.882727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:23.891520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:23.892097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:23.892378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:23.892637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:23.902653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:23.907345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:23.908501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:24:23.909480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 584/1180] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:23.914770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:23.943195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:24.016447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab08ba9-a5a6-40b6-a1d9-74dbe1748d87 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:24.051104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:24.051728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:24.052005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:24.052272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:24.052934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:24.053232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:24.054613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:24.062736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 597/1181] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:24:24.068973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:24.174016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a297-69c2-470e-a3d8-de862de49918 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:24.208667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:24.209298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:24.209570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:24.209833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] PATH_INFO: `/v3/users/945251b6e28a4dde9dcb78f11dc174fd` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:24.223407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] RBAC: Authorizing `identity:get_user(user_id=945251b6e28a4dde9dcb78f11dc174fd)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:24.224871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:24.226506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 585/1182] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:24 2022] GET /identity/v3/users/945251b6e28a4dde9dcb78f11dc174fd => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:24.232989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:24.326183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:24.337129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab1a74-8f7a-4315-9847-c0d805710325 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:24.371405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:24.372007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:24.372285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:24.372552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:24.449164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:24.450578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:24.539880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:24.551108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 598/1183] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:24:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:24:24.557496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:24.605884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:24.669752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74947d53-4d6f-4342-91e9-a0cf89458b0f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:24.706557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:24.707111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:24.707413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:24.707687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] PATH_INFO: `/v3/projects/69591aa2514e476e8af15a077b77bd2b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:24.708868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] RBAC: Authorizing `identity:get_project(project_id=69591aa2514e476e8af15a077b77bd2b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:24.710315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbc7177-7761-4fa8-a82b-7a22e2c4a2e2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:24.712011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 586/1184] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:24 2022] GET /identity/v3/projects/69591aa2514e476e8af15a077b77bd2b => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:24:24.718621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:24.822096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514ffdb-394f-4f47-9f83-e5bd8d018cd0 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:24.857591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:24.858193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:24.858465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:24.858741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:24.859507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=69591aa2514e476e8af15a077b77bd2b, user.id=945251b6e28a4dde9dcb78f11dc174fd) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:24.859817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:24.861255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735f982-6ce1-49ab-9a55-c9b81afec3b0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:24.866774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 599/1185] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:24:24 2022] GET /identity/v3/role_assignments?user.id=945251b6e28a4dde9dcb78f11dc174fd&scope.project.id=69591aa2514e476e8af15a077b77bd2b&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:26.831611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacf9f8-04af-4292-9b58-38ed86e08e5e None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:26.832232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacf9f8-04af-4292-9b58-38ed86e08e5e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:26.832232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baacf9f8-04af-4292-9b58-38ed86e08e5e None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:26.833522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 587/1186] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:26.839141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:26.839558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:26.839710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:26.859197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:26.860450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:26.913024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:26.932050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 600/1187] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:26.954150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacf9f8-04af-4292-9b58-38ed86e08e5e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:27.061718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baacf9f8-04af-4292-9b58-38ed86e08e5e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:27.076725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:27.099259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:27.099812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:27.100136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:27.100428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:27.112746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:27.117839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:27.118972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:24:27.119984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 588/1188] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:27.125292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:27.230392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d5225-3e7d-4ee5-ac44-b7768410fde6 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:27.266196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:27.266799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:27.267071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:27.267372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:27.268031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:27.268329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:27.269701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:27.277811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 601/1189] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:27.283894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:27.395862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ef562-8153-4e14-b798-329d84b1d1ec None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:27.437998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:27.438637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:27.438922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:27.439194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] PATH_INFO: `/v3/users/945251b6e28a4dde9dcb78f11dc174fd` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:27.440413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] RBAC: Authorizing `identity:get_user(user_id=945251b6e28a4dde9dcb78f11dc174fd)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:27.441887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:27.443542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 589/1190] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:27 2022] GET /identity/v3/users/945251b6e28a4dde9dcb78f11dc174fd => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:27.450265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:27.501673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:27.561694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040f920-250d-4f40-b1bf-193057903eeb None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:27.598442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:27.599072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:27.599384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:27.599653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] PATH_INFO: `/v3/projects/69591aa2514e476e8af15a077b77bd2b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:27.600935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] RBAC: Authorizing `identity:get_project(project_id=69591aa2514e476e8af15a077b77bd2b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:27.602376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f2b8e-a80b-4162-99f3-ce4b8481dd9f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:27.604076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 602/1191] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:27 2022] GET /identity/v3/projects/69591aa2514e476e8af15a077b77bd2b => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:27.610550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:27.656716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:27.718140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be66e30-085b-485b-a49b-9fad0af8316e None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:27.755751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:27.756415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:27.756689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:27.756959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] PATH_INFO: `/v3/projects/69591aa2514e476e8af15a077b77bd2b/users/945251b6e28a4dde9dcb78f11dc174fd/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:27.758955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] RBAC: Authorizing `identity:create_grant(project_id=69591aa2514e476e8af15a077b77bd2b, user_id=945251b6e28a4dde9dcb78f11dc174fd, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:27.760441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02874842-c434-433e-9f8e-be3652ebc771 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:27.774529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 590/1192] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:24:27 2022] PUT /identity/v3/projects/69591aa2514e476e8af15a077b77bd2b/users/945251b6e28a4dde9dcb78f11dc174fd/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:24:29.734583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2b2d-8f2b-4919-855d-36607583fd10 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:29.735180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2b2d-8f2b-4919-855d-36607583fd10 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:29.735180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2b2d-8f2b-4919-855d-36607583fd10 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:29.736505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 603/1193] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:24:29.742190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a547-a998-48cd-88ba-368443615041 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:29.742612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a547-a998-48cd-88ba-368443615041 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:29.742727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e757a547-a998-48cd-88ba-368443615041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:29.764109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e757a547-a998-48cd-88ba-368443615041 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:24:29.766128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e757a547-a998-48cd-88ba-368443615041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:24:29.837463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 591/1194] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:24:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:24:29.861317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d2b2d-8f2b-4919-855d-36607583fd10 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:29.986455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d2b2d-8f2b-4919-855d-36607583fd10 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.001048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:30.025528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.026168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.026448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.026722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.038713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:24:30.043433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:24:30.044560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:24:30.045405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 604/1195] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:24:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:24:30.051189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757a547-a998-48cd-88ba-368443615041 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:30.084016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:30.164919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e757a547-a998-48cd-88ba-368443615041 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.202009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.202637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.202920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.203221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.203897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:30.204195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:30.205558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:30.215574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 592/1196] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:24:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:30.221934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:30.329577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc99aa4-0df0-40d3-8453-9492a6656967 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.366350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.367018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.367335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.367610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] PATH_INFO: `/v3/users/945251b6e28a4dde9dcb78f11dc174fd` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.368863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] RBAC: Authorizing `identity:get_user(user_id=945251b6e28a4dde9dcb78f11dc174fd)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:30.370346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:30.372067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 605/1197] 10.222.0.33 () {56 vars in 1177 bytes} [Mon Aug 8 13:24:30 2022] GET /identity/v3/users/945251b6e28a4dde9dcb78f11dc174fd => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:30.378473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:30.482442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:30.490837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3587355-6529-4abf-bcfa-82a81cd649be None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.529289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.529904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.530182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.530453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.600856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:30.602254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:30.695520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:30.703624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 593/1198] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:24:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:24:30.709978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:30.755666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:30.813829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce98f57c-e2d1-4680-b1b1-3f5b7aa375a9 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.849236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.849871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.850144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.850419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] PATH_INFO: `/v3/projects/69591aa2514e476e8af15a077b77bd2b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.851682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] RBAC: Authorizing `identity:get_project(project_id=69591aa2514e476e8af15a077b77bd2b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:30.853062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f2b4ec-4454-4bf3-b05c-f7927acbbf9a None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:30.854718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 606/1199] 10.222.0.33 () {56 vars in 1186 bytes} [Mon Aug 8 13:24:30 2022] GET /identity/v3/projects/69591aa2514e476e8af15a077b77bd2b => generated 361 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:30.861211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:24:30.960567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bd4a61-7fe2-43b2-b9ae-0dfaf7db0a10 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:24:30.993112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:24:30.993724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:24:30.993993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:24:30.994241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:24:30.994963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=69591aa2514e476e8af15a077b77bd2b, user.id=945251b6e28a4dde9dcb78f11dc174fd) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:24:30.995294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:24:30.996731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a28fc3-d5ac-4780-8a05-dbdb0e5b758b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:24:31.002541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 594/1200] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:24:30 2022] GET /identity/v3/role_assignments?user.id=945251b6e28a4dde9dcb78f11dc174fd&scope.project.id=69591aa2514e476e8af15a077b77bd2b&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:24:40.844246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=45 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:24:40.999853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=71 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:26.704936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ac4ea-dbe0-48a0-96f1-5190300d8ad6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:26.705571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ac4ea-dbe0-48a0-96f1-5190300d8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:26.705571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ac4ea-dbe0-48a0-96f1-5190300d8ad6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:26.707109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 607/1201] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:26.712782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:26.713329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:26.713329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:26.735311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:26.736606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:26.825107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 595/1202] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 08 13:25:26.850714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ac4ea-dbe0-48a0-96f1-5190300d8ad6 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:27.002426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ac4ea-dbe0-48a0-96f1-5190300d8ad6 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:27.054966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:27.056012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:27.056414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:27.056673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:27.070383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:27.072100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:27.091201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:27.092401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:27.093405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:27.093405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:27.093405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:27.093405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 608/1203] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:27.100592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:27.243108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d614a9-cb72-4c3e-a7b8-ad8253e075fc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:27.296030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:27.301234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:27.301983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:27.302896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:27.303250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:27.304057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:27.304396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:27.306310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d42f658-1497-415c-8fc8-1aef7e249542 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:27.321680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 596/1204] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:27.328550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:27.411604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:27.479143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba610b4-a8c3-41b8-9e5f-84a398ae0ffe None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:27.530698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:27.531638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:27.532014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:27.532378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:27.533165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:27.534766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55d38a-f22a-4d05-a081-20255a5df258 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:27.568976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 609/1205] 10.222.0.33 () {60 vars in 1132 bytes} [Mon Aug 8 13:25:27 2022] POST /identity/v3/users => generated 251 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:25:29.929139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d66b-3663-462d-9f16-d6ba02903a27 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:29.929806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d66b-3663-462d-9f16-d6ba02903a27 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:29.929806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc37d66b-3663-462d-9f16-d6ba02903a27 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:29.931326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 597/1206] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:29.939962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:29.940511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:29.940616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:29.963055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:29.964365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:30.055965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 610/1207] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:30.094667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d66b-3663-462d-9f16-d6ba02903a27 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:30.261048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc37d66b-3663-462d-9f16-d6ba02903a27 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:30.314570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:30.315328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:30.315608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:30.315891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:30.329353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:30.334742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:30.336939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:30.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:30.341532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 598/1208] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:25:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:30.342726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:30.409397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:30.501432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cececf0-9a3c-4597-b748-5824d42037a7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:30.557403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:30.558104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:30.558448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:30.558724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:30.559523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:30.559856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:30.561394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:30.572428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 611/1209] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:25:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:25:30.578409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:30.652494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:30.740883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6de31c6-74be-464f-abe1-592b1a1bd762 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:30.806513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:30.806513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:30.806513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:30.806513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:30.818220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:30.823482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:30.838082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:30.840177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:30.842082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:30.842082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:30.842082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:30.843340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 599/1210] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:30.846119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:31.039297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:31.044007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe411e4-a88b-4948-854b-e8f00da28d21 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:31.099449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:31.100262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:31.100754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:31.101133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:31.102060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:31.102452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:31.104367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:31.121904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 612/1211] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:31.130122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:31.278061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b1e577-7dc8-4669-8fed-2d2d08ed53b8 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:31.280077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:31.320993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:31.321871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:31.322107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:31.322400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:31.332685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:31.334299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:31.347830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:31.350253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:31.352319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 600/1212] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:31.356622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:31.534052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089301d8-f4e4-4768-ad5f-37368c152e51 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:31.594710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:31.595705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:31.596152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:31.596641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:31.598880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:31.598880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:31.600763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:31.632336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 613/1213] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:25:31.642357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:31.814580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84547b7a-ef36-48a9-aa2a-cbc88a2e22ec None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:31.872345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:31.873532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:31.874164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:31.874843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:31.937314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:32.018451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:32.020994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:32.177041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 601/1214] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:25:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:25:32.183222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:32.211505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:32.370329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885daf9-54b0-449d-a9d5-7c8b6679a4e4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:32.431771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:32.432325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:32.432604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:32.432875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:32.445002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:32.446562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:32.462535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:32.464776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:32.466720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:32.466720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:32.466720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:32.466720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 614/1215] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:32.469948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:32.582945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:32.658947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f5959-addc-413c-a8aa-8c24b685e70b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:32.724046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:32.724959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:32.725434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:32.725900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:32.727006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:32.727568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:32.730104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:32.752930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 602/1216] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:25:32.762968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:32.912317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:32.981567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffad8ba-add5-449c-a020-8b74509b0f86 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:33.040750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:33.041815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:33.042281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:33.042734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:33.060657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:33.064258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:33.092792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:33.094029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:33.095062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:33.104967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 615/1217] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:33.105183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:33.289987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa9428-d1a6-4bd3-9004-85ab58bcc648 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:33.338110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:33.359364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:33.360246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:33.360863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:33.360863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:33.361596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:33.361925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:33.363578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628e285-58b1-450a-9b24-d422fb8f131c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:33.380396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 603/1218] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:33.387457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:33.521986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde7435d-04bf-4b51-bd37-feb5195bdc9f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:33.557604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:33.567135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:33.567811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:33.568251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:33.568708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:33.569662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=44983e1fc3fc4079a31d1e078fb475fe) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:33.569983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:33.572327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3c8477-38a7-4d94-abae-efb9abbd0bcb None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:33.580034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 616/1219] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:25:33 2022] GET /identity/v3/role_assignments?user.id=44983e1fc3fc4079a31d1e078fb475fe&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:25:35.867790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb22eb-e37a-4ba4-b3ca-e7d832e28af9 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:35.868424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb22eb-e37a-4ba4-b3ca-e7d832e28af9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:35.868424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb22eb-e37a-4ba4-b3ca-e7d832e28af9 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:35.870018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 604/1220] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:35.876442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:35.877405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:35.877503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:35.900112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:35.901561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:36.007569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 617/1221] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:36.039175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb22eb-e37a-4ba4-b3ca-e7d832e28af9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:36.183097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb22eb-e37a-4ba4-b3ca-e7d832e28af9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:36.233700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:36.235022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:36.235744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:36.236236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:36.256759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:36.262152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:36.263368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:36.264271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 605/1222] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:25:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:36.270687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:36.417624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e68490c-e13a-4ac2-81d2-384d1324cbc9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:36.463686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:36.481903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:36.482905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:36.483411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:36.483854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:36.485099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:36.485571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:36.487375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:36.500774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 618/1223] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:25:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:36.508770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:36.570141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:36.642743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ca2d0-e723-419d-8d41-9746745c5db1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:36.684348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:36.685107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:36.685516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:36.685974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:36.700265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:36.701953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:36.723790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:36.724807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:36.725823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:36.725823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:36.725823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:36.725823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 606/1224] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:36.733380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:36.976142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d7a2c-bfa2-4a52-8df2-0e974b7558c1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:37.028213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:37.029247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:37.030019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:37.030486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:37.031786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:37.032313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:37.035359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:37.061197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 619/1225] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:37.068369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:37.206028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:37.215818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a1a64-0f4d-4618-ac72-0a4e190718f1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:37.273932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:37.274661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:37.275050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:37.275688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:37.286165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:37.289344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:37.304869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:37.307248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:37.309180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 607/1226] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:37.314757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:37.393418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:37.480822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076da34-370e-4851-a81c-b7205771b5e4 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:37.532561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:37.533256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:37.533646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:37.534036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:37.535067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:37.535455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:37.537732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:37.559825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 620/1227] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:37.566316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:37.699399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef201a9-52bd-4adf-87d9-a0cfa5590ea1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:37.751446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:37.752991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:37.753558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:37.753953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:37.772078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:37.773837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:37.794684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:37.796452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:37.797929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:37.797929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:37.797929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:37.797929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 608/1228] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:37.804505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:37.949531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e27ed-df5d-4956-a59c-28a0c1fb8b6e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:37.952504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:37.995414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:37.996728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:37.996728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:37.996728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:37.997411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:37.997712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:37.999126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:38.013777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 621/1229] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:38.020366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:38.032832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:38.183753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10681de2-4c4d-4a75-ad5d-cf59849a9aff None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:38.247887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:38.248643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:38.249156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:38.249860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:38.261394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:38.263739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:38.281265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:38.283557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:38.284602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 609/1230] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:38.294663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:38.467130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1562e-b036-4ae7-8619-f8db044d817f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:38.525579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:38.526529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:38.527412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:38.527853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:38.528943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:38.529413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:38.532070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f73731-bc28-402a-81fe-ecb8d6c44705 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:38.550999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 622/1231] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:38.558993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:38.666812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:38.719800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93965a31-3a32-486d-b3ed-ba5ed300d1a0 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:38.762401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:38.763120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:38.763404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:38.763677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/44983e1fc3fc4079a31d1e078fb475fe/roles/5154aa0f817548bf891f739dab185a54` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:38.782628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=44983e1fc3fc4079a31d1e078fb475fe, role_id=5154aa0f817548bf891f739dab185a54)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:38.784526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032c7663-f7a7-42f0-a15a-b5006d1a71e5 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:38.800423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 610/1232] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:25:38 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/44983e1fc3fc4079a31d1e078fb475fe/roles/5154aa0f817548bf891f739dab185a54 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:25:41.146867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8575c-5488-4412-9752-e6bf67aa7f7b None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:41.147783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8575c-5488-4412-9752-e6bf67aa7f7b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:41.147969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8575c-5488-4412-9752-e6bf67aa7f7b None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:41.149866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 623/1233] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:41.156583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:41.157270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:41.157270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:41.179148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:41.181231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:41.298066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 611/1234] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:41.323252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8575c-5488-4412-9752-e6bf67aa7f7b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:41.375353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:41.477933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8575c-5488-4412-9752-e6bf67aa7f7b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:41.518760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:41.519450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:41.519731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:41.520010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:41.532632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:41.537669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:41.544928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:41.545854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 624/1235] 10.222.0.33 () {56 vars in 1102 bytes} [Mon Aug 8 13:25:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:41.547033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:41.606496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:41.695037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150bf7b7-ac68-4d76-bbbc-9f738a9969a5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:41.752257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:41.752955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:41.753347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:41.753750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:41.754586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] RBAC: Adding query filter params (name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:41.754900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:41.756575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:41.770443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 612/1236] 10.222.0.33 () {56 vars in 1103 bytes} [Mon Aug 8 13:25:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:41.777820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:41.914898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:41.938267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2353d561-9f06-4833-89e2-4439ef4c4567 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:41.983978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:41.984831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:41.985243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:41.985665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:42.001040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:42.004400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:42.025565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:42.027596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:42.028913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:42.028913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:42.028913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:42.028913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 625/1237] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:42.037152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:42.180475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7757a453-d7d2-4473-8c4f-ee20570e014d None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:42.192671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:42.236673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:42.237723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:42.238201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:42.238688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:42.239907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:42.240390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:42.242766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:42.259724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 613/1238] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:42.267400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:42.396915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c869bf-034e-48db-bd15-9656ecf33937 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:42.449494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:42.450300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:42.450712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:42.451136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:42.455756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:42.463404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:42.465876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:42.482973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:42.485350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:42.494676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 626/1239] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:42.495423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:42.643735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88765548-5877-4834-90b9-312ceefd8422 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:42.704078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:42.704957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:42.705627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:42.706189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:42.707809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:42.708352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:42.711139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:42.739442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 614/1240] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:42.747097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:42.916509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9dfd-d434-49b7-b448-92860f988bb2 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:42.968561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:42.969638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:42.970044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:42.970476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:43.081700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:43.095765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:43.098541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:43.265529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 627/1241] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:25:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 08 13:25:43.274804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:43.323725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:43.442518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2a78d-45db-4ca1-ba17-b1d8ba22e2a2 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:43.500925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:43.502365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:43.502851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:43.503348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:43.524009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:43.527489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:43.554897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:43.556157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:43.557046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:43.557046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:43.557046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:43.557046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 615/1242] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:43.563131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:43.708551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:43.716051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bd495e-0ffe-44f5-aeba-562adf42ab6f None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:43.761534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:43.762158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:43.762436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:43.762706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:43.763411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:43.763711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:43.765171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:43.778126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 628/1243] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:43.785373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:43.920118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:43.953296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913cd5f-a5c0-4dcf-b38c-f3ba1b47ea53 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:43.994554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:43.995624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:43.995932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:43.996212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:44.004763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:44.006420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:44.022898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:44.024223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:44.025685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 616/1244] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:44.031604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:44.158158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ea656-da88-4d89-b8f1-b17ba8895044 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:44.198299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:44.199048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:44.199394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:44.199682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:44.200552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:44.200868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:44.202596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60241199-e96a-4301-85b2-2ec85942ff7b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:44.218158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 629/1245] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:44.225721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:44.384483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91480d2-d8f5-43fd-b7de-590741085a50 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:44.427454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:44.428186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:44.428467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:44.428770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:44.429501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] RBAC: Adding query filter params (role.id=5154aa0f817548bf891f739dab185a54, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=44983e1fc3fc4079a31d1e078fb475fe) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:44.434363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:44.434363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d59d48-751b-443b-9e27-9116fdde55b9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:44.438200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 617/1246] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:25:44 2022] GET /identity/v3/role_assignments?user.id=44983e1fc3fc4079a31d1e078fb475fe&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=5154aa0f817548bf891f739dab185a54 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:46.609612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9971-4fc4-44e4-bb85-372b18fb666a None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:46.610417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9971-4fc4-44e4-bb85-372b18fb666a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:46.610417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9971-4fc4-44e4-bb85-372b18fb666a None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:46.611856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 630/1247] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:46.617931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:46.618583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:46.618583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:46.627785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:46.645200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:46.646374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:46.744360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 618/1248] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:46.768291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9971-4fc4-44e4-bb85-372b18fb666a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:46.789721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:46.915803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9971-4fc4-44e4-bb85-372b18fb666a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:46.962457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:46.963559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:46.964058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:46.964547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:46.983513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:46.992984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:46.994053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:46.994885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:46.998454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 631/1249] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:47.009312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:47.175008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc06b7-af39-4588-b1b5-f592b644fa6b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:47.225362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:47.232917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:47.233755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:47.234097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:47.234467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:47.235302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:47.235697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:47.237375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:47.250901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 619/1250] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:25:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:47.259632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:47.369130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:47.404658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c486118-fca9-4088-baeb-6a50bd1bf28e None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:47.453472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:47.454031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:47.454303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:47.454661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:47.466637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:47.468344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:47.486913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:47.489000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:47.491020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:47.491020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:47.491020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:47.491020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 632/1251] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:47.500015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:47.621836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3261e5-47f5-442e-827e-88eb27b6e98b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:47.664869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:47.665704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:47.665957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:47.666352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:47.666983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:47.667336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:47.668831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:47.682777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 620/1252] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:47.694442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:47.859589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e7cd-e044-48f0-be60-e7dd627b51a6 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:47.909431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:47.931501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:47.932372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:47.932713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:47.933061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:47.944348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:47.946446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:47.961795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:47.963533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:47.964669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 633/1253] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:47.970687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:48.051108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:48.124264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e657d91-194b-4d79-a1bc-d6bba2a956a9 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:48.173496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:48.174477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:48.174820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:48.175030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:48.176193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:48.176680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:48.178719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:48.203958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 621/1254] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:48.212315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:48.373723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272d632-32e6-4f8a-bcaf-5551a8fa8456 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:48.414095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:48.414777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:48.415115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:48.415397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:48.502114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:48.519877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:48.521791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:48.656637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 634/1255] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:25:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:25:48.662719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:48.811951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d98f86-cccb-4411-bffd-1a8807fdeb2f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:48.828847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:48.861646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:48.862349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:48.862666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:48.863209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:48.877113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:48.878894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:48.899109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:48.901233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:48.903071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:48.903071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:48.903071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:48.903071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 622/1256] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:48.908988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:49.024261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:49.047375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b396b2c-b8f5-4cee-ac3a-00405f6afb9d None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:49.093525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:49.094145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:49.094426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:49.094692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:49.095402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:49.095705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:49.097199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:49.119496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 635/1257] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:49.127410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:49.263171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96fcfa2-d239-4921-8f10-7c611bd8b2ae None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:49.310951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:49.311752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:49.312124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:49.312498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:49.319955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:49.321445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:49.333102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:49.334265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:49.335737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 623/1258] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:49.340316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:49.473371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037cb1c-6bcf-4a0f-88bd-c6f7063ce396 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:49.515453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:49.515971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:49.516254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:49.516529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:49.517237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:49.517546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:49.519006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a8a48-850c-45b1-a957-de293467b181 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:49.539603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 636/1259] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:49.547438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:49.580904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:49.680854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6626e7-1c69-4ed9-b89e-9ad51232b73c None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:49.728401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:49.729382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:49.729871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:49.730326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:49.731375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=44983e1fc3fc4079a31d1e078fb475fe) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:49.731812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:49.733951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17e3d4-d016-4147-b55d-9fd231a02056 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:49.743435 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 624/1260] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:25:49 2022] GET /identity/v3/role_assignments?user.id=44983e1fc3fc4079a31d1e078fb475fe&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:51.981309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d461241-485a-45ad-b3ab-4caabbecb9e7 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:51.981988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d461241-485a-45ad-b3ab-4caabbecb9e7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:51.981988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d461241-485a-45ad-b3ab-4caabbecb9e7 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:51.983390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 637/1261] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:51.989005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:51.989543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:51.989543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:52.010804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:52.012178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:52.087521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 625/1262] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:52.111433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d461241-485a-45ad-b3ab-4caabbecb9e7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:52.123741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:52.231205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d461241-485a-45ad-b3ab-4caabbecb9e7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:52.268896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:52.269463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:52.269799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:52.270067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:52.281661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:52.286314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:52.287533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:52.288659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 638/1263] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:25:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:52.294056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:52.312958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:52.401266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929766c7-a6c7-4fcb-84bf-1429c6e503ca None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:52.442229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:52.442850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:52.443151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:52.443425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:52.444098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:52.444394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:52.445786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:52.456077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 626/1264] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:25:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:52.464349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:52.542276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:52.589676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa6ca59-3a3c-41aa-8588-6a9d1e0437a1 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:52.628290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:52.629182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:52.629611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:52.629999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:52.644316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:52.647180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:52.674453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:52.675770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:52.676894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:52.676894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:52.676894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:52.676894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 639/1265] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:52.683973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:52.772628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:52.792256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a761c43-c2f5-4746-94c5-99a7c3ae404f None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:52.827292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:52.827791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:52.828077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:52.828343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:52.829001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:52.829299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:52.830684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:52.843527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 627/1266] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:52.849938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:52.976440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f963efd3-0edd-401b-bea4-1199219a7921 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.001165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:53.018435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.019262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.019684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.019984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.028440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.030011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:53.042804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:53.044081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:53.045287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 640/1267] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 08 13:25:53.050928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:53.166637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721819d8-93c3-4e25-a50b-60d9afe8ab4b None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.208056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.208681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.208956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.209226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.209975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:53.210274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.211769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.221866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:53.234098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 628/1268] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:53.240802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:53.366295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa0f69-2245-4c7e-9ee9-b488da022847 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.409694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.410530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.410898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.411237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.423120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.424639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:53.440198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:53.442127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:53.444361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:53.444361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:53.444361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:53.444361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 641/1269] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:53.449106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:53.559413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519fea82-275a-43b0-a93d-4e23586609f7 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.599664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.600305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.600633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.600969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.601692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:53.601994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.603629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.616458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 629/1270] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:53.622362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:53.656694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:53.734748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdbc6b-4510-428d-8a8f-ba33110f7f31 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.776093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.776957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.777235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.777511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.786377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.789054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:53.804201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:53.805394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:53.806375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 642/1271] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:53.811510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:53.876161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:53.935323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a825dc5e-c8a9-45b1-9444-435bc8819f62 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:53.973668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:53.974388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:53.974669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:53.974931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:53.975832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:53.978704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:53.978704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdd0efb-1fa6-40ff-a52e-9e3e235e0cac None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:53.991453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 630/1272] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:53.997964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:54.104437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:54.113612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf39a5a-6472-473a-b0a1-0900d8f4ba12 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:54.148306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:54.148961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:54.149236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:54.149540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] PATH_INFO: `/v3/projects/27dd5e0f9c7b47598842644c14231836/users/44983e1fc3fc4079a31d1e078fb475fe/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:54.151561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] RBAC: Authorizing `identity:create_grant(project_id=27dd5e0f9c7b47598842644c14231836, user_id=44983e1fc3fc4079a31d1e078fb475fe, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:54.153199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b44b6-0b5a-49e5-8d05-8ef2734ae88c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:54.167200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 643/1273] 10.222.0.33 () {58 vars in 1439 bytes} [Mon Aug 8 13:25:53 2022] PUT /identity/v3/projects/27dd5e0f9c7b47598842644c14231836/users/44983e1fc3fc4079a31d1e078fb475fe/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:25:56.227253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf19-5b2e-468b-a293-63a0b9d43ef7 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:56.227824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf19-5b2e-468b-a293-63a0b9d43ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:56.227949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf19-5b2e-468b-a293-63a0b9d43ef7 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:56.229920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 631/1274] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:25:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:25:56.237550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:56.238022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:56.238342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:56.265195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:25:56.266964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:25:56.363184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 644/1275] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:25:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:25:56.387179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2adf19-5b2e-468b-a293-63a0b9d43ef7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:56.499464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2adf19-5b2e-468b-a293-63a0b9d43ef7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:56.518609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:56.543514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:56.544154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:56.544428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:56.544699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:56.556316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 08 13:25:56.561593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:56.562800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 08 13:25:56.563742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:56.567282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 632/1276] 10.222.0.33 () {56 vars in 1096 bytes} [Mon Aug 8 13:25:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:56.569093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:56.661629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:56.682016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bce8595-91c1-4b5f-96d0-25933cc38ea5 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:56.717508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:56.718129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:56.718409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:56.718678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:56.719394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:56.719690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:56.721084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:56.731552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 645/1277] 10.222.0.33 () {56 vars in 1099 bytes} [Mon Aug 8 13:25:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:56.738226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:56.846468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded198d8-8801-466b-a002-e18cd04cf484 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:56.882464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:56.883132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:56.883412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:56.883676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:56.895876 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:56.897433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:56.914359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:56.915689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:56.916576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:56.916576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:56.916576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:56.916576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 633/1278] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:56.923328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:57.091347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc5ea-5d7a-481b-b623-4c482f891891 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:57.144403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:57.145321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:57.145754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:57.146249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:57.147154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:57.147616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:57.149902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:57.169499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 646/1279] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:57.176197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:57.216975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:57.297235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12621316-5a4e-4e8d-96f1-7b50e1680474 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:57.350623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:57.351299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:57.351581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:57.351852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:57.360669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:57.362466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 08 13:25:57.375490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:57.377133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:25:57.378147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 634/1280] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:57.383773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:57.388319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:57.502668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e4dde5-e54d-40da-b7f5-73624254360b None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:57.543825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:57.544500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:57.544788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:57.545079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:57.545968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:57.546268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:57.547744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:57.568013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 647/1281] 10.222.0.33 () {56 vars in 1143 bytes} [Mon Aug 8 13:25:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:25:57.575030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:57.706001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:57.718052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53350606-1bae-4c61-af2d-5c3aa8cfbac6 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:57.761504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:57.762093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:57.762421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:57.762740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:57.840284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:57.841798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:57.949027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:57.963645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 635/1282] 10.222.0.33 () {58 vars in 1282 bytes} [Mon Aug 8 13:25:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 08 13:25:57.977582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:58.055730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:58.100576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cc9aa5-91b7-419a-a02b-8697f624a326 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:58.140049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:58.140902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:58.141264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:58.141573 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:58.157814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:58.159798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 08 13:25:58.178368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:58.180050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 08 13:25:58.181841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 08 13:25:58.181841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 08 13:25:58.181841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:58.181841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 648/1283] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:25:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:58.186819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:58.291049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc052c0-f595-4171-ae6b-a181f99228a1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:58.326315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:58.327035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:58.327341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:58.327681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:58.328548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:58.328919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:58.330511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:58.342922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 636/1284] 10.222.0.33 () {56 vars in 1109 bytes} [Mon Aug 8 13:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:58.352768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:58.464286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074ef3e-ddf6-40cf-85f0-c392275a3544 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:58.478925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:58.504589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:58.505340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:58.505646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:58.505959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:58.514694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:58.516390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 08 13:25:58.528186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application with Lock( Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:25:58.529322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: ERROR keystone.server.flask.application Aug 08 13:25:58.530332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 649/1285] 10.222.0.33 () {56 vars in 1111 bytes} [Mon Aug 8 13:25:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:25:58.535925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:58.593691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:25:58.647888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669b2467-4f62-4c69-b7fd-1d9a7117de00 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:58.682334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:58.682982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:58.683283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:58.683555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:58.684323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:58.684621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:58.686329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce9d26c-6c42-4688-a62a-ca3fbdacd916 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:58.699254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 637/1286] 10.222.0.33 () {56 vars in 1148 bytes} [Mon Aug 8 13:25:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:25:58.706662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:25:58.816311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c406fb7-6b80-4ccb-a105-7a9f8d529d21 None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:25:58.850696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:25:58.851259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:25:58.851574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:25:58.852111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:25:58.852562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] RBAC: Adding query filter params (role.id=193a5d91676042449492edcffd7d1760, scope.project.id=27dd5e0f9c7b47598842644c14231836, user.id=44983e1fc3fc4079a31d1e078fb475fe) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:25:58.852860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:25:58.854361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678769b-2ce3-4c48-a148-f23016c50b4e None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:25:58.860536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 650/1287] 10.222.0.33 () {56 vars in 1374 bytes} [Mon Aug 8 13:25:58 2022] GET /identity/v3/role_assignments?user.id=44983e1fc3fc4079a31d1e078fb475fe&scope.project.id=27dd5e0f9c7b47598842644c14231836&role.id=193a5d91676042449492edcffd7d1760 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:00.919277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca943-367f-4c41-83ef-d9b41c98dfb2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:00.919956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca943-367f-4c41-83ef-d9b41c98dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:00.919956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264ca943-367f-4c41-83ef-d9b41c98dfb2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:00.921262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 638/1288] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:26:00.928203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:00.929312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:00.929697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:00.956224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:26:00.958099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:26:00.996764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:01.039780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 651/1289] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:26:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:26:01.062432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ca943-367f-4c41-83ef-d9b41c98dfb2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:01.180567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264ca943-367f-4c41-83ef-d9b41c98dfb2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:01.191834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:01.215224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:01.215855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:01.216128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:01.216392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:01.216969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:01.218380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:26:01.226962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:26:01.228239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:01.229391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 639/1290] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:26:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:26:01.235625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:01.346057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8cf3ae-db61-4ae9-bee1-f010a0da6363 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:01.380799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:01.381409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:01.381689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:01.381962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:01.382606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:26:01.382905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:01.384442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d64baea-98e5-4939-acaa-ee303735c944 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:01.390186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 652/1291] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 13:26:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:01.396782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:01.507370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ca305e-d629-4143-bbf6-d9af3bdfb496 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:01.546107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:01.546717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:01.546994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:01.547275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] PATH_INFO: `/v3/services` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:01.547906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:26:01.548207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:01.549594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fad11-de98-44fb-bdeb-736fc7910f67 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:01.554306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 640/1292] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 13:26:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:03.542581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788be39-f75e-430d-8ca5-9b7654dbc9ce None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:03.543528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788be39-f75e-430d-8ca5-9b7654dbc9ce None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:03.543528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788be39-f75e-430d-8ca5-9b7654dbc9ce None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:03.545050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 653/1293] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:26:03.550570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5006a8b-c021-489b-b11e-b067b0a5eaaf None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:03.550984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5006a8b-c021-489b-b11e-b067b0a5eaaf None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:03.551135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5006a8b-c021-489b-b11e-b067b0a5eaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:03.570198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b5006a8b-c021-489b-b11e-b067b0a5eaaf None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:26:03.571436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b5006a8b-c021-489b-b11e-b067b0a5eaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:26:03.608442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:03.649913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 641/1294] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:26:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:26:03.672276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788be39-f75e-430d-8ca5-9b7654dbc9ce None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:03.705943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:03.781089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788be39-f75e-430d-8ca5-9b7654dbc9ce None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:03.822372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:03.822995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:03.823306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:03.823577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:03.824228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:03.825603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6172608-cd6b-4286-937b-c3c1b7b4afa0 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:03.855211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 654/1295] 10.222.0.33 () {60 vars in 1141 bytes} [Mon Aug 8 13:26:03 2022] POST /identity/v3/services => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:26:05.844650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d4280-0d96-4361-b737-24f8e4dcf33d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:05.845201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d4280-0d96-4361-b737-24f8e4dcf33d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:05.845201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d4280-0d96-4361-b737-24f8e4dcf33d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:05.846766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 642/1296] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:26:05.853146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:05.853603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:05.853789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:05.873211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:26:05.874395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:26:05.967618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 655/1297] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:26:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:26:05.992425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d4280-0d96-4361-b737-24f8e4dcf33d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:06.098037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d4280-0d96-4361-b737-24f8e4dcf33d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:06.148546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:06.149291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:06.149596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:06.149855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:06.150674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:06.152192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:26:06.161971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:26:06.163781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:06.165506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 643/1298] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:26:05 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:26:06.170665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:06.190684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:06.280086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a72428-a695-49ca-ba29-886e207c4e90 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:06.313284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:06.313888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:06.314166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:06.314429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:06.315053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:26:06.315387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:06.316794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:06.322018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 656/1299] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 13:26:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:06.328291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:06.359598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:06.431009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5476a3c-088c-4ec3-93a8-3390266f9fe4 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:06.464807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:06.465413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:06.465690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:06.465947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:06.466655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] RBAC: Adding query filter params (interface=public, service_id=8402b4da9cf54900ba8f64c7edb4852e, region_id=RegionOne) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:26:06.466965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:06.468385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce120a4a-1dea-425a-81d4-1f1563aedfa0 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:06.473431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 644/1300] 10.222.0.33 () {56 vars in 1251 bytes} [Mon Aug 8 13:26:06 2022] GET /identity/v3/endpoints?service_id=8402b4da9cf54900ba8f64c7edb4852e&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:06.479589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:06.573728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:06.599631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54599d9-20d7-46a0-8988-05e013bc6e2c None admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:06.637596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:06.638102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:06.638376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:06.638750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:06.639370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:06.640732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591441f6-e071-4254-9c85-4532feabf853 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:06.647938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 657/1301] 10.222.0.33 () {56 vars in 1087 bytes} [Mon Aug 8 13:26:06 2022] GET /identity/v3/services => generated 2191 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:26:08.617785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0d3df-2725-4acc-b146-7230be0eb511 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:08.618335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0d3df-2725-4acc-b146-7230be0eb511 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:08.618335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0d3df-2725-4acc-b146-7230be0eb511 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:08.619917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 645/1302] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:26:08.625974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:08.626610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:08.626737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:08.647596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:26:08.649003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:26:08.726859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 658/1303] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:26:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 08 13:26:08.750676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0d3df-2725-4acc-b146-7230be0eb511 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:08.850554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:08.858032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a0d3df-2725-4acc-b146-7230be0eb511 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:08.892227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:08.892862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:08.893138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:08.893415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:08.894002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:08.895422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 08 13:26:08.904774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 08 13:26:08.905954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.get_or_create( Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application with Lock( Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self._enter() Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application return self.creator() Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application created_value = creator( Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: ERROR keystone.server.flask.application Aug 08 13:26:08.906829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 646/1304] 10.222.0.33 () {56 vars in 1117 bytes} [Mon Aug 8 13:26:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 08 13:26:08.912164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:08.990672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:09.014749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426387a-2574-4e37-9a93-b8d863857d12 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:09.050140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:09.050748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:09.051004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:09.051339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] PATH_INFO: `/v3/services` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:09.051982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:26:09.052279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:09.053648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8955d-3256-4d98-9a2a-a20c83c4d709 None admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:09.058917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 659/1305] 10.222.0.33 () {56 vars in 1116 bytes} [Mon Aug 8 13:26:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:26:09.065679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:26:09.181729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fc82-f659-4fa3-b48c-e276695963b1 None admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:26:09.222558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:26:09.223209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:26:09.223486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:26:09.223753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:26:09.224436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:26:09.225816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699a284-0b7c-4e79-b2b6-7df0d129c576 None admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:26:09.248022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 647/1306] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:26:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:26:19.056280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=30 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:26:19.243273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:15.170108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bc20d-1321-40d4-a878-2160c2979dc4 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:15.170564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bc20d-1321-40d4-a878-2160c2979dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:15.171061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0bc20d-1321-40d4-a878-2160c2979dc4 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:15.179465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 660/1307] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:15.185217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ae089-9f29-4769-b418-93a4b7874c1b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:15.185752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ae089-9f29-4769-b418-93a4b7874c1b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:15.186187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ae089-9f29-4769-b418-93a4b7874c1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:15.320689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-178ae089-9f29-4769-b418-93a4b7874c1b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:15.322921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-178ae089-9f29-4769-b418-93a4b7874c1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:15.462321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 648/1308] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:15.516433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:15.516826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:15.517105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:15.518834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 661/1309] 10.222.0.33 () {54 vars in 965 bytes} [Mon Aug 8 13:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:15.526476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f6f7-cc30-45b7-8117-78bcc4ca8a75 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:15.527201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f6f7-cc30-45b7-8117-78bcc4ca8a75 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:15.527616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660f6f7-cc30-45b7-8117-78bcc4ca8a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:15.630536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c660f6f7-cc30-45b7-8117-78bcc4ca8a75 None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:15.631807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c660f6f7-cc30-45b7-8117-78bcc4ca8a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:15.659712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:15.762017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 649/1310] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Aug 8 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:15.774939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:15.777351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:15.956392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f45d82-d2d8-4a7d-887c-fafe226a88f2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:16.083788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:16.085313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:16.086012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:16.086648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:16.089040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:16.183380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:16.282952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:16.285784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46798703-95bf-4edb-a06c-af006c4a48b3 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:16.317903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 662/1311] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:19.008147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:19.008784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:19.008784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:19.010075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 650/1312] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:19.016085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd746-5002-47f2-abd3-b2074299a309 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:19.016701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd746-5002-47f2-abd3-b2074299a309 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:19.016701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdfd746-5002-47f2-abd3-b2074299a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:19.113392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-fbdfd746-5002-47f2-abd3-b2074299a309 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:19.115550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fbdfd746-5002-47f2-abd3-b2074299a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:19.246865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 663/1313] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:19.285931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:19.287535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:19.402370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170dd6f8-2c2b-4709-91c6-bae7fd55ab5f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:19.477026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:19.480891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:19.481534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:19.481822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:19.482097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:19.484248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:19.605902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:19.607371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a32b-4b27-4d74-a9e1-b3faca97853e service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:19.628347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 651/1314] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:21.991199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:21.992070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:21.992070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:21.993146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 664/1315] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:27:22.000295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bb9d6-09f5-432e-9dc3-fb95645e3f47 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:22.000852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bb9d6-09f5-432e-9dc3-fb95645e3f47 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:22.001239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bb9d6-09f5-432e-9dc3-fb95645e3f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:22.095064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4d1bb9d6-09f5-432e-9dc3-fb95645e3f47 None None] MFA Rules not processed for user `0415b7f9c30e4b5880842ee378935c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:22.096391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4d1bb9d6-09f5-432e-9dc3-fb95645e3f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:22.160078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:22.243748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 652/1316] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:22.285558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:22.287252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:22.298041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:22.408099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7237c-61b4-4ff7-a5d2-f87952248358 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:22.485678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:22.486293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:22.486553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:22.486833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:22.488519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:22.559766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:22.613437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:22.614910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e722d1-5f80-4791-8c5c-a2da9c169b16 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:22.635919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 665/1317] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:25.378905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:25.379667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:25.379667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:25.382084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 653/1318] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:27:25.386897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f18f1-6501-4ce1-b307-096cf7fb6f22 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:25.387478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f18f1-6501-4ce1-b307-096cf7fb6f22 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:25.387478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f18f1-6501-4ce1-b307-096cf7fb6f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:25.469141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-863f18f1-6501-4ce1-b307-096cf7fb6f22 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:25.470436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-863f18f1-6501-4ce1-b307-096cf7fb6f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:25.588062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 666/1319] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:25.628488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:25.630210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:25.745873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a028c8-a558-49c0-9796-85da902ce5f1 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:25.823899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:25.824510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:25.824786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:25.825056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:25.826691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:25.834696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:25.945692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:25.947225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0de2-b035-45fa-a021-01579e7c69c6 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:25.968198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 654/1320] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:28.719491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:28.720073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:28.720073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:28.721410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 667/1321] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:28.726671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d85d-0ec0-4b5d-862b-355b0144634c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:28.727121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d85d-0ec0-4b5d-862b-355b0144634c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:28.727390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4471d85d-0ec0-4b5d-862b-355b0144634c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:28.796630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4471d85d-0ec0-4b5d-862b-355b0144634c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:28.797872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4471d85d-0ec0-4b5d-862b-355b0144634c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:28.857388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:28.905248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 655/1322] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:28.943716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:28.945304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:28.967112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:29.068217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0054853-90a1-4a26-81ff-b4476a104956 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:29.149749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:29.150379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:29.150658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:29.150931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:29.152611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:29.231108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:29.269764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:29.271266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3f361-21cb-47a8-81c9-a64372f7f3a3 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:29.290625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 668/1323] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:31.431904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:31.432529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:31.432529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:31.433827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 656/1324] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:31.439818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aa730-c6f7-4a3f-9e32-e5aaf11a9379 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:31.440351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aa730-c6f7-4a3f-9e32-e5aaf11a9379 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:31.440351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393aa730-c6f7-4a3f-9e32-e5aaf11a9379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:31.511415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-393aa730-c6f7-4a3f-9e32-e5aaf11a9379 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:31.512643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-393aa730-c6f7-4a3f-9e32-e5aaf11a9379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:31.611983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 669/1325] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:31.648722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:31.650337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:31.759782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92012cbf-9daf-44f3-810f-7fc4fc92fb79 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:31.840925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:31.841536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:31.841811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:31.842060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:31.843710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:31.862366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:31.959428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:31.960922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0829f86d-ba1c-4330-b692-a9421bac6f99 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:31.980133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 657/1326] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:34.111998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:34.112593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:34.112593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:34.114043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 670/1327] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:34.119659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9265-7a9c-4bd9-82d6-bf8d093f27cc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:34.120164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9265-7a9c-4bd9-82d6-bf8d093f27cc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:34.120164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6f9265-7a9c-4bd9-82d6-bf8d093f27cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:34.195645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8a6f9265-7a9c-4bd9-82d6-bf8d093f27cc None None] MFA Rules not processed for user `0415b7f9c30e4b5880842ee378935c5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:34.197034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8a6f9265-7a9c-4bd9-82d6-bf8d093f27cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:34.268646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:34.301830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 658/1328] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:34.339190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:34.340778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:34.377485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:34.464811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cbc15-84f8-4ad7-906a-9b4ea94c3d13 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:34.541483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:34.542128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:34.542402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:34.542691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:34.544349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:34.645809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:34.683362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:34.684851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b972003-bced-4cd9-8030-b4a02c507b6e service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:34.705729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 671/1329] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:37.103246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:37.103246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:37.103738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:37.104705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 659/1330] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:37.110411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c5b5-79c0-4d4d-9fc0-f1305730d45f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:37.110867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c5b5-79c0-4d4d-9fc0-f1305730d45f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:37.110973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b6c5b5-79c0-4d4d-9fc0-f1305730d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:37.187825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a9b6c5b5-79c0-4d4d-9fc0-f1305730d45f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:37.189075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a9b6c5b5-79c0-4d4d-9fc0-f1305730d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:37.298041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 672/1331] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:37.332810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:37.334380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:37.459797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f94fa0-0d14-4f5d-bc0e-4f45b093b32d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:37.546078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:37.546720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:37.547508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:37.547783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:37.550293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:37.584498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:37.672301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:37.673883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737bd6f1-9875-47f5-9948-d2d8d8da91e4 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:37.693413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 660/1332] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:39.786008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9503-a309-4820-98c4-16403ddb5cd5 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:39.786624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9503-a309-4820-98c4-16403ddb5cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:39.786624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d9503-a309-4820-98c4-16403ddb5cd5 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:39.787941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 673/1333] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:39.793612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3d2a2-4130-45f5-a11c-a63c67e40d0d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:39.794066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3d2a2-4130-45f5-a11c-a63c67e40d0d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:39.794270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3d2a2-4130-45f5-a11c-a63c67e40d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:39.874419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a7e3d2a2-4130-45f5-a11c-a63c67e40d0d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:39.875681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a7e3d2a2-4130-45f5-a11c-a63c67e40d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:39.961863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:39.986061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 661/1334] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:40.039342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:40.039604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:40.039798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:40.041332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 674/1335] 10.222.0.33 () {54 vars in 965 bytes} [Mon Aug 8 13:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:40.047920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c925-c08e-4616-99e2-019a41fe2ada None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:40.048208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c925-c08e-4616-99e2-019a41fe2ada None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:40.048566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d54c925-c08e-4616-99e2-019a41fe2ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:40.146741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6d54c925-c08e-4616-99e2-019a41fe2ada None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:40.148020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6d54c925-c08e-4616-99e2-019a41fe2ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:40.232195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 662/1336] 10.222.0.33 () {58 vars in 1062 bytes} [Mon Aug 8 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:40.241795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:40.243439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:40.291194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:40.370743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d025cb9-8dab-451e-87e5-e9e9e6db70d4 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:40.455294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:40.455937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:40.456217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:40.456486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:40.458293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:40.561930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:40.592778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:40.594264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3b2f0-f13c-4832-b9b4-9f13371cb834 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:40.615058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 675/1337] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:43.525104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:43.525722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:43.525722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:43.527029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 663/1338] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:43.532554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e291b-b066-4035-b6fe-189061a28090 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:43.533100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e291b-b066-4035-b6fe-189061a28090 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:43.533100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e291b-b066-4035-b6fe-189061a28090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:43.603339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f85e291b-b066-4035-b6fe-189061a28090 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:43.604611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f85e291b-b066-4035-b6fe-189061a28090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:43.712159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 676/1339] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:43.749135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:43.750695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:43.833130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:43.861968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba5c0-6894-458e-a813-1dda51d2f3ea None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:43.939068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:43.939724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:43.940036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:43.940312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:43.941973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:44.066843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:44.068349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17523a9-e7d9-4b57-b139-446b151193da service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:44.213182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 664/1340] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:48.337052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:48.337658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:48.337658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:48.338961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 677/1341] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:48.344677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd94f-0716-4b10-961d-0566a1290377 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:48.345153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd94f-0716-4b10-961d-0566a1290377 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:48.345153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd94f-0716-4b10-961d-0566a1290377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:48.351185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:48.426121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1f1fd94f-0716-4b10-961d-0566a1290377 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:48.428234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1f1fd94f-0716-4b10-961d-0566a1290377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:48.527985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 665/1342] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:48.566937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:48.568707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:48.625589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:48.699631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951a10b-a8d1-401d-82eb-581113d36ca1 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:48.794470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:48.795107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:48.795394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:48.795683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:48.797322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:48.898578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:48.918924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:48.920603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742aac86-1384-47c5-b731-afad8ad1cd89 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:48.941366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 678/1343] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:51.672673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:51.673272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:51.673272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:51.674581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 666/1344] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:51.680108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b22b-d6c3-4ba2-a986-4e5c8e2048ba None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:51.680574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b22b-d6c3-4ba2-a986-4e5c8e2048ba None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:51.680821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f62b22b-d6c3-4ba2-a986-4e5c8e2048ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:51.752744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0f62b22b-d6c3-4ba2-a986-4e5c8e2048ba None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:51.754002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0f62b22b-d6c3-4ba2-a986-4e5c8e2048ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:51.864516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 679/1345] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:51.902621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:51.904203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:52.002645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:52.021840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4e158-9f36-44c4-9323-a0d9ec7fa685 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:52.100432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:52.101045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:52.101340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:52.101609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:52.103514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:52.249318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:52.250812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba770b63-bfff-4c20-bcde-021fd501d673 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:52.270650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 667/1346] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:27:56.444934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:56.445532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:56.445532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:56.447144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 680/1347] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:27:56.453235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8df2-5e67-4607-af9a-86c8d398145e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:56.453702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8df2-5e67-4607-af9a-86c8d398145e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:56.453789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d8df2-5e67-4607-af9a-86c8d398145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:56.471880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:56.533592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a90d8df2-5e67-4607-af9a-86c8d398145e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:27:56.534941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a90d8df2-5e67-4607-af9a-86c8d398145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:56.673172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 668/1348] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:27:56.718486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:27:56.720389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:56.792953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:56.852259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcd311-4c83-470c-98b1-573cbc9a5a85 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:27:56.945530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:27:56.946271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:27:56.946570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:27:56.946846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:27:56.948636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:27:57.080552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:27:57.086592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:27:57.088176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe876e1-0bf4-47ad-9695-32bba1b8df9c service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:27:57.109499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 681/1349] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:02.323138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce430ed-e140-4c5c-b03d-8f9706075dd2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:02.323990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce430ed-e140-4c5c-b03d-8f9706075dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:02.324429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce430ed-e140-4c5c-b03d-8f9706075dd2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:02.325990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 669/1350] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:28:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:02.333190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5493f-943c-4feb-b73b-27e6771817a5 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:02.333694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5493f-943c-4feb-b73b-27e6771817a5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:02.333808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c5493f-943c-4feb-b73b-27e6771817a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:02.455401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b1c5493f-943c-4feb-b73b-27e6771817a5 None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:02.455401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b1c5493f-943c-4feb-b73b-27e6771817a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:02.659320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 682/1351] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:28:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:06.670986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=59 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:06.882229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed13bd-db90-4174-81d5-246fcaf6a3c7 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:06.882229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed13bd-db90-4174-81d5-246fcaf6a3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:06.882229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed13bd-db90-4174-81d5-246fcaf6a3c7 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:06.883998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 670/1352] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:06.890323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446c2e-15b2-4e28-b948-67b469785b4b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:06.890846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446c2e-15b2-4e28-b948-67b469785b4b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:06.891295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a446c2e-15b2-4e28-b948-67b469785b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:06.968181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:06.994990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0a446c2e-15b2-4e28-b948-67b469785b4b None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:06.996353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0a446c2e-15b2-4e28-b948-67b469785b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:07.146193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 683/1353] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:08.557170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f7480-7a48-474c-b9d9-07c7c92f9ca2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:08.557729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f7480-7a48-474c-b9d9-07c7c92f9ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:08.557958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f7480-7a48-474c-b9d9-07c7c92f9ca2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:08.559544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 671/1354] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:08.565837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990789bd-0e3b-443b-a4a2-bd1d56abc45d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:08.566456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990789bd-0e3b-443b-a4a2-bd1d56abc45d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:08.566456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990789bd-0e3b-443b-a4a2-bd1d56abc45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:08.659483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-990789bd-0e3b-443b-a4a2-bd1d56abc45d None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:08.661423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-990789bd-0e3b-443b-a4a2-bd1d56abc45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:08.720378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:08.795938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 684/1355] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:12.586762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee321-64c3-44d4-b6ee-4ad96b2183c9 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:12.587450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee321-64c3-44d4-b6ee-4ad96b2183c9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:12.587633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94ee321-64c3-44d4-b6ee-4ad96b2183c9 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:12.590086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 672/1356] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:12.598138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227d304-6078-4562-9dae-25e6c9c345f7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:12.599000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227d304-6078-4562-9dae-25e6c9c345f7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:12.599223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5227d304-6078-4562-9dae-25e6c9c345f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:12.679844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5227d304-6078-4562-9dae-25e6c9c345f7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:12.681995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5227d304-6078-4562-9dae-25e6c9c345f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:12.882371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 685/1357] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:12.989833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:12.990773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:12.993043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:12.995947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 673/1358] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 13:28:12 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:13.005526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6366065-6d89-4aea-9c7c-2c442a1ff7b2 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:13.007360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6366065-6d89-4aea-9c7c-2c442a1ff7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:13.009655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6366065-6d89-4aea-9c7c-2c442a1ff7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:13.050585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:13.166125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b6366065-6d89-4aea-9c7c-2c442a1ff7b2 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:13.168205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b6366065-6d89-4aea-9c7c-2c442a1ff7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:13.420810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 686/1359] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 13:28:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:13.435622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:13.438900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:13.760897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a803c8-f42a-4934-858f-fbe5ad19a769 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:13.899869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:13.996184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:14.002311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:14.007322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:14.008278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:14.012627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:14.205144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:14.209819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9e8e7e-ea91-412f-884c-3031f5207998 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:14.253097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 674/1360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 818 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:17.004815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:17.005590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:17.005590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:17.006932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 687/1361] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:28:17.012534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc3f40-5041-46a8-8191-78e7ec287d55 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:17.013162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc3f40-5041-46a8-8191-78e7ec287d55 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:17.013162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc3f40-5041-46a8-8191-78e7ec287d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:17.094685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-aecc3f40-5041-46a8-8191-78e7ec287d55 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:17.096147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-aecc3f40-5041-46a8-8191-78e7ec287d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:17.147516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:17.226256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 675/1362] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:28:17.274394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:17.276168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:17.403338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:17.527837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ad5cbf-857e-4fca-b170-51eed11096a5 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:17.723949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:17.724753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:17.725641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:17.726090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:17.729349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:17.762705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:17.851099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:17.852893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e1ca6-1b9f-4aa5-9088-c3e4ba682681 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:17.872731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 688/1363] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:20.399215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd9fb-dd1b-4fcd-b3e3-e894154cbc38 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:20.399800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd9fb-dd1b-4fcd-b3e3-e894154cbc38 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:20.400091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29dd9fb-dd1b-4fcd-b3e3-e894154cbc38 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:20.402481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 676/1364] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:20.408327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d892-e7fc-4db8-981a-c201e3f9bafc None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:20.408813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d892-e7fc-4db8-981a-c201e3f9bafc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:20.408935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911d892-e7fc-4db8-981a-c201e3f9bafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:20.484397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2911d892-e7fc-4db8-981a-c201e3f9bafc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:20.485625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2911d892-e7fc-4db8-981a-c201e3f9bafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:20.612235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:20.617200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 689/1365] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:20.674338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:20.674798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:20.675297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:20.677713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 677/1366] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 13:28:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:20.686263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1715-b45a-45e0-bb0d-fab18a0cf523 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:20.686793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1715-b45a-45e0-bb0d-fab18a0cf523 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:20.687354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e1715-b45a-45e0-bb0d-fab18a0cf523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:20.794803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-904e1715-b45a-45e0-bb0d-fab18a0cf523 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:20.796358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-904e1715-b45a-45e0-bb0d-fab18a0cf523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:20.977660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 690/1367] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 13:28:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:20.987929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:20.989682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:21.183222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743a048-2e0a-460c-9bc2-6830be35276a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:21.207340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:21.341964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:21.342862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:21.343371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:21.343855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:21.347049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:21.524936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:21.528164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973acd0c-86d8-455d-8b08-dd50b2a00b0f service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:21.564317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 678/1368] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:22.510287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898937a-c8a0-475e-a5c2-ef85bff3a4df None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:22.511011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898937a-c8a0-475e-a5c2-ef85bff3a4df None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:22.511011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a898937a-c8a0-475e-a5c2-ef85bff3a4df None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:22.512515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 691/1369] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:28:22.517813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789d76-8da8-489b-98b5-05fd47ae90e8 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:22.518072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789d76-8da8-489b-98b5-05fd47ae90e8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:22.518350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56789d76-8da8-489b-98b5-05fd47ae90e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:22.555937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:22.601298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-56789d76-8da8-489b-98b5-05fd47ae90e8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:22.602526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-56789d76-8da8-489b-98b5-05fd47ae90e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:22.705554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 679/1370] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:23.021839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:23.022529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:23.023123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:23.026241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 692/1371] 10.222.0.33 () {54 vars in 964 bytes} [Mon Aug 8 13:28:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:23.035283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7052-24f9-4271-81b2-274c9352e621 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:23.035948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7052-24f9-4271-81b2-274c9352e621 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:23.036517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb7052-24f9-4271-81b2-274c9352e621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:23.128636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-82fb7052-24f9-4271-81b2-274c9352e621 None None] MFA Rules not processed for user `656cb98cf9b147f993f2cd31ec4ea55c`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:23.130184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-82fb7052-24f9-4271-81b2-274c9352e621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:23.186060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:23.220452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 680/1372] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Aug 8 13:28:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:23.229292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:23.232894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:23.359388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:23.458300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f8da-c35a-4790-b35d-6d031b4dda14 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:23.562978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:23.564070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:23.564444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:23.564879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:23.566846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:23.677967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:23.696835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:23.698465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a9031-e9b3-4f71-b20f-779c6ec46ce3 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:23.722425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 693/1373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:24.313438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:24.314959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:24.315266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:24.317772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 681/1374] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:28:24.323959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d3e4-3c3b-4960-bca0-b6866f401285 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:24.324352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d3e4-3c3b-4960-bca0-b6866f401285 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:24.324678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa6d3e4-3c3b-4960-bca0-b6866f401285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:24.413577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-daa6d3e4-3c3b-4960-bca0-b6866f401285 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:24.414986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-daa6d3e4-3c3b-4960-bca0-b6866f401285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:24.519915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 694/1375] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:28:24.570980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:24.572693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:24.743990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139aca9-1c64-423d-9f33-332d0412c42c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:24.798031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:24.868414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:24.869012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:24.869645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:24.870003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:24.871834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:24.971009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:24.972656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00688678-5a4b-4064-a19f-1431ba255e28 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:24.992571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 682/1376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:27.168714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:27.169243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:27.169342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:27.170748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 695/1377] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:27.176634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5b40-255d-4e1d-9517-c0bb1ad9e2c8 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:27.177108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5b40-255d-4e1d-9517-c0bb1ad9e2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:27.177272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f5b40-255d-4e1d-9517-c0bb1ad9e2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:27.249295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-591f5b40-255d-4e1d-9517-c0bb1ad9e2c8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:27.250527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-591f5b40-255d-4e1d-9517-c0bb1ad9e2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:27.257528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:27.360751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 683/1378] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:27.414604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:27.417272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:27.519347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:27.657507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7753a58-449a-4c8d-aa63-b8aa161e0de9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:27.791710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:27.792326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:27.792664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:27.792944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:27.794559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:27.824117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:27.940081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:27.945347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450edfcc-11cb-45c1-82ff-0651ec54f7ee service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:27.975476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 696/1379] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:30.103494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:30.104105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:30.104105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:30.105655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 684/1380] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:30.111066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4c528-a56b-401f-8ad4-57680699e2d6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:30.111645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4c528-a56b-401f-8ad4-57680699e2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:30.111645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a4c528-a56b-401f-8ad4-57680699e2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:30.200644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-26a4c528-a56b-401f-8ad4-57680699e2d6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:30.202401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-26a4c528-a56b-401f-8ad4-57680699e2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:30.317993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:30.333468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 697/1381] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:30.375060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:30.375370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:30.546966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:30.560181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4064a0-01a3-447b-98d9-96cc30c9c6c8 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:30.708009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:30.709526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:30.709813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:30.710088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:30.712709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:30.820447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:30.822106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daa4e78-0add-4d8d-a867-1a72fcd7be28 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:30.834395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:30.851388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 685/1382] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:33.276455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:33.277366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:33.277366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:33.279278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 698/1383] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:33.289410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32baf0-7706-45d0-bdd4-517e3f83bbee None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:33.290285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32baf0-7706-45d0-bdd4-517e3f83bbee None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:33.290285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32baf0-7706-45d0-bdd4-517e3f83bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:33.398927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9c32baf0-7706-45d0-bdd4-517e3f83bbee None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:33.400328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9c32baf0-7706-45d0-bdd4-517e3f83bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:33.581026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 686/1384] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:33.638453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:33.640275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:33.856193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31aa80-3ac1-434a-8573-3c7e5f877fbb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:33.960796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:34.036797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:34.037515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:34.037842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:34.038214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:34.040271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:34.164524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:34.166963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901e2117-4a8c-4df1-9cc0-0b7712d63171 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:34.198420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 699/1385] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:36.499857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:36.500945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:36.500945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:36.503327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 687/1386] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:36.509999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb2a6-2c9c-4edc-83ac-a09947992fbc None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:36.510920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb2a6-2c9c-4edc-83ac-a09947992fbc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:36.510920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78cb2a6-2c9c-4edc-83ac-a09947992fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:36.635710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a78cb2a6-2c9c-4edc-83ac-a09947992fbc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:36.636907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a78cb2a6-2c9c-4edc-83ac-a09947992fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:36.680757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:36.773320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 700/1387] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:36.829883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:36.831559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:36.942075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:37.042229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba793e5-2fb2-46a1-a2a4-148cf9cb8824 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:37.223743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:37.226010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:37.226300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:37.226576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:37.229144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:37.294524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:37.367905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:37.370338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8fc009-fbc7-425a-a0ae-a3799f3751f0 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:37.397943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 688/1388] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:39.439398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029c9e-2e6b-41f3-ac2c-1667a4141676 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.440199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029c9e-2e6b-41f3-ac2c-1667a4141676 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.440199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029c9e-2e6b-41f3-ac2c-1667a4141676 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.441268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 701/1389] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:28:39.448706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1cd0-7bcd-4888-8f9d-3e365a2cc35c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.449304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1cd0-7bcd-4888-8f9d-3e365a2cc35c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.449665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302e1cd0-7bcd-4888-8f9d-3e365a2cc35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.561990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-302e1cd0-7bcd-4888-8f9d-3e365a2cc35c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:39.563307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-302e1cd0-7bcd-4888-8f9d-3e365a2cc35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:39.697502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 689/1390] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:39.765556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483de0-c8dc-4fc7-a4d8-d64a3992a128 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.766127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483de0-c8dc-4fc7-a4d8-d64a3992a128 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.766677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c483de0-c8dc-4fc7-a4d8-d64a3992a128 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.769288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 702/1391] 10.222.0.33 () {54 vars in 967 bytes} [Mon Aug 8 13:28:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:39.778491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.778975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.779499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.802850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:39.874966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] MFA Rules not processed for user `f034fd79c9ef454498ee0874d2c97d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:39.876213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:39.955430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9a23-958f-4436-96e8-27fd5f3eef31 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.955741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9a23-958f-4436-96e8-27fd5f3eef31 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.955947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fa9a23-958f-4436-96e8-27fd5f3eef31 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.957335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 703/1392] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:28:39.963121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:39.963363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:39.963668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:39.983584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 690/1393] 10.222.0.33 () {58 vars in 1064 bytes} [Mon Aug 8 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:39.991667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:40.000770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ac093-45bf-46e9-9daf-4ce8a306df54 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:40.049207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:40.050626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:40.108966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:40.109371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:40.109738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:40.109885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:40.111593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:40.152492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:40.171478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 704/1394] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:28:40.226663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:40.228944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:40.250683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:40.260609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:40.262578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5e99a1-5ae3-42bd-b815-891375ba288c service glance] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:40.286624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 691/1395] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:40.412718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb97e1-fe4b-4f59-a121-759ed7799486 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:40.536160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:40.536955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:40.536955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:40.537183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:40.539059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:40.550480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:40.686454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:40.689896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98399e4a-b808-4357-bca1-85cb6e8139b3 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:40.722611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 705/1396] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:41.241755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db479-bcca-484d-9265-d177f7dd40de None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:41.242414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db479-bcca-484d-9265-d177f7dd40de None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:41.242414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211db479-bcca-484d-9265-d177f7dd40de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:41.339822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-211db479-bcca-484d-9265-d177f7dd40de None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:41.341981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-211db479-bcca-484d-9265-d177f7dd40de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:41.478138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:41.485910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 692/1397] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:41.547550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:41.548139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:41.548139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:41.550449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 706/1398] 10.222.0.33 () {54 vars in 943 bytes} [Mon Aug 8 13:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:41.556962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c30d-bceb-4dbb-a1e4-21e46f079b77 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:41.557382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c30d-bceb-4dbb-a1e4-21e46f079b77 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:41.557734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359c30d-bceb-4dbb-a1e4-21e46f079b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:41.662544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c359c30d-bceb-4dbb-a1e4-21e46f079b77 None None] MFA Rules not processed for user `ec66fef9dc274223b34659e369063ced`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:41.664044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c359c30d-bceb-4dbb-a1e4-21e46f079b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:41.840483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 693/1399] 10.222.0.33 () {58 vars in 1040 bytes} [Mon Aug 8 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:41.852626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:41.854861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:42.065355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:42.068437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcaef-ea8a-416a-ab7b-011041c400b9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:42.202157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:42.202930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:42.203359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:42.203692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:42.205760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:42.316722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:42.318513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c660a6-4133-49a5-9a4a-a28f3fd8a407 service swift] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:42.329355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 707/1400] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:28:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:28:43.000807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:43.001690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:43.001690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:43.002729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 694/1401] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:43.008772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133f08-2fd5-49fd-83ee-4b101d46c560 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:43.009352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133f08-2fd5-49fd-83ee-4b101d46c560 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:43.009352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76133f08-2fd5-49fd-83ee-4b101d46c560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:43.069158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:43.107889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-76133f08-2fd5-49fd-83ee-4b101d46c560 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:43.109633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-76133f08-2fd5-49fd-83ee-4b101d46c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:43.245700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 708/1402] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:28:43.296036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:43.297675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:43.372964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:43.498986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97e42e-07c5-4b28-81f9-d4665ab53fd2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:43.645533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:43.646094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:43.646372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:43.646639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:43.648406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:43.671291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:43.801436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:43.803395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1c386-4eac-43b8-af22-551233dd57ea service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:43.824998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 695/1403] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:28:46.076967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:46.077975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:46.078313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:46.080004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 709/1404] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:46.085482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381924ae-d316-49c4-b5ed-9e019f965d5b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:46.085992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381924ae-d316-49c4-b5ed-9e019f965d5b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:46.085992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381924ae-d316-49c4-b5ed-9e019f965d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:46.171660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-381924ae-d316-49c4-b5ed-9e019f965d5b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:46.173616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-381924ae-d316-49c4-b5ed-9e019f965d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:46.284178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:46.293335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 696/1405] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:46.340174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:46.341780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:46.470872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:46.523678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac07d4b-35e7-4cac-b9d9-f634930c6116 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:46.659503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:46.660121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:46.660397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:46.660667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:46.662287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:46.760457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:46.779281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:46.780999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1689c8-d856-4a5f-ac19-b4acdee0bd35 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:46.804305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 710/1406] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:48.878407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:48.879026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:48.879026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:48.881513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 697/1407] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:28:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:28:48.887346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a996417-fc0c-4135-a93a-fe6217ea4c05 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:48.887898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a996417-fc0c-4135-a93a-fe6217ea4c05 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:48.887898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a996417-fc0c-4135-a93a-fe6217ea4c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:48.971261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3a996417-fc0c-4135-a93a-fe6217ea4c05 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:28:48.972528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3a996417-fc0c-4135-a93a-fe6217ea4c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:49.115993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 711/1408] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:28:49.165521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:28:49.167071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:49.370444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad41999f-eb03-46e0-927d-237017847f28 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:28:49.466131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:49.524966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:28:49.525589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:28:49.525873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:28:49.526144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:28:49.527998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:28:49.656197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:28:49.658489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d78988-d282-401a-a5ff-37c3da2a7bf8 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:28:49.685059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 698/1409] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:28:59.114168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=78 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:28:59.678793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=65 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:05.518703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff6df-294c-4738-8c5e-9f093305b9d6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:05.519487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff6df-294c-4738-8c5e-9f093305b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:05.519487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ff6df-294c-4738-8c5e-9f093305b9d6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:05.520653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 712/1410] 10.222.0.33 () {54 vars in 918 bytes} [Mon Aug 8 13:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:05.526998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76872d8-0eae-4426-b0e5-e387fce41ef6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:05.527515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76872d8-0eae-4426-b0e5-e387fce41ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:05.527515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76872d8-0eae-4426-b0e5-e387fce41ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:05.613529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f76872d8-0eae-4426-b0e5-e387fce41ef6 None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:05.614759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f76872d8-0eae-4426-b0e5-e387fce41ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:05.733214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 699/1411] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 8 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:05.748520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70674b-bb35-4b7d-af26-37763d137a6c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:05.748826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70674b-bb35-4b7d-af26-37763d137a6c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:05.749094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70674b-bb35-4b7d-af26-37763d137a6c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:05.750508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 713/1412] 10.222.0.33 () {54 vars in 918 bytes} [Mon Aug 8 13:29:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:05.755586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6fb3-1f78-4506-9648-d9fb5aaa73d9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:05.755872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6fb3-1f78-4506-9648-d9fb5aaa73d9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:05.756144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfe6fb3-1f78-4506-9648-d9fb5aaa73d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:05.845679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:05.848701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2bfe6fb3-1f78-4506-9648-d9fb5aaa73d9 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:05.850759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2bfe6fb3-1f78-4506-9648-d9fb5aaa73d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:05.969406 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 700/1413] 10.222.0.33 () {58 vars in 1015 bytes} [Mon Aug 8 13:29:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:08.492337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:08.492935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:08.492935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:08.494431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:08.504007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63275bc-ecc1-4ab3-90a5-9bf10c57ed15 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:08.504571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63275bc-ecc1-4ab3-90a5-9bf10c57ed15 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:08.504571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63275bc-ecc1-4ab3-90a5-9bf10c57ed15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:08.587494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b63275bc-ecc1-4ab3-90a5-9bf10c57ed15 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:08.588729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b63275bc-ecc1-4ab3-90a5-9bf10c57ed15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:08.649941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:08.697925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:08.743406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:08.745121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:08.930685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f66b8e4-316e-40be-953b-313d9ba15aef None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:09.013593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:09.060578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:09.061202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:09.061484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:09.061759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:09.063468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:09.166102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:09.167781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767fbb24-da97-4964-b824-9022e3895882 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:09.186790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 715/1416] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:10.115900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:10.116522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:10.116522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:10.117828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:10.123493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89bc61-7812-425a-8b0f-426d15f4caa6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:10.124135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89bc61-7812-425a-8b0f-426d15f4caa6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:10.124135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89bc61-7812-425a-8b0f-426d15f4caa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:10.206026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6c89bc61-7812-425a-8b0f-426d15f4caa6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:10.207345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6c89bc61-7812-425a-8b0f-426d15f4caa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:10.236367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:10.309030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:10.460678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:10.462259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:10.611412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd49887-2205-4a8f-a235-fce6513a9536 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:10.656600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:10.732965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:10.733925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:10.734209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:10.734475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:10.736145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:10.827946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:10.829563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107e5472-93a0-408b-8b83-0754e030f640 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:10.847995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 703/1419] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:11.747281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89668276-2f78-4c17-9b10-44da772cbade None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:11.747922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89668276-2f78-4c17-9b10-44da772cbade None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:11.747922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89668276-2f78-4c17-9b10-44da772cbade None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:11.749214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:11.754465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01099c8f-391e-4002-99cc-bbd849ca3440 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:11.754953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01099c8f-391e-4002-99cc-bbd849ca3440 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:11.755056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01099c8f-391e-4002-99cc-bbd849ca3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:11.811033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:11.834880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-01099c8f-391e-4002-99cc-bbd849ca3440 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:11.836142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-01099c8f-391e-4002-99cc-bbd849ca3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:11.937979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:12.098910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89668276-2f78-4c17-9b10-44da772cbade None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:12.100468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-89668276-2f78-4c17-9b10-44da772cbade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:12.264174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89668276-2f78-4c17-9b10-44da772cbade None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:12.277137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:12.386012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:12.386632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:12.386907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:12.387230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:12.388843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:12.488300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:12.489963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e9cf9-901d-46f5-a92b-f717aab34746 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:12.509262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 718/1422] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:13.364195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:13.364756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:13.364756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:13.366154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:13.372962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c58-ca62-456f-b927-c9b59aa2d381 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:13.373470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c58-ca62-456f-b927-c9b59aa2d381 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:13.373470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0048c58-ca62-456f-b927-c9b59aa2d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:13.396742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:13.459218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-d0048c58-ca62-456f-b927-c9b59aa2d381 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:13.460414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d0048c58-ca62-456f-b927-c9b59aa2d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:13.569739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:13.720599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:13.722100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:13.853199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:13.897401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e3642-05dd-4f26-a26e-964b1d2a67d2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:14.035703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:14.036344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:14.036793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:14.037070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:14.038850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:14.132048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:14.150254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:14.152046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aec585d-60b3-4206-9cd0-0ee22fe77197 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:14.171844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 706/1425] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:15.035336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:15.035936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:15.035936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:15.037253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:15.043132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1581-0550-4ba3-be03-916fde6e2987 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:15.043572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1581-0550-4ba3-be03-916fde6e2987 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:15.043690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1581-0550-4ba3-be03-916fde6e2987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:15.126976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-70bc1581-0550-4ba3-be03-916fde6e2987 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:15.128246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-70bc1581-0550-4ba3-be03-916fde6e2987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:15.230061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:15.379718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:15.381275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:15.471102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:15.538809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15a4f0-7547-48e1-b44d-06243f16bbbf None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:15.656581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:15.657195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:15.657470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:15.657739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:15.659382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:15.710224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:15.756924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:15.758554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853925d5-ef70-4794-ac3a-5f280d968b7d service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:15.777329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 721/1428] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:16.582774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:16.583500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:16.583500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:16.585122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:16.591124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcfb4-6f57-40a5-b31d-4b7475103c07 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:16.592210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcfb4-6f57-40a5-b31d-4b7475103c07 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:16.592210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bcfb4-6f57-40a5-b31d-4b7475103c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:16.672166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-827bcfb4-6f57-40a5-b31d-4b7475103c07 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:16.673436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-827bcfb4-6f57-40a5-b31d-4b7475103c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:16.785835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:16.936734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:16.938331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:16.995938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:17.097345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e4667-f5a8-402d-ab2d-3dcae8242475 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:17.221010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:17.221570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:17.221845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:17.222111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:17.223767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:17.231849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:17.316631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:17.318246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d35fd-7a72-4f26-8a69-6290b36df54f service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:17.336095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 709/1431] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:18.144070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:18.144709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:18.144709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:18.145975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:18.151643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4911-4d99-48f7-94c5-6fc87db6438e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:18.152204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4911-4d99-48f7-94c5-6fc87db6438e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:18.152204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb4911-4d99-48f7-94c5-6fc87db6438e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:18.233732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d8cb4911-4d99-48f7-94c5-6fc87db6438e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:18.234971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d8cb4911-4d99-48f7-94c5-6fc87db6438e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:18.308632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:18.342136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:18.511505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:18.513041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:18.531526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:18.672395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fb963b-e4e2-4c19-90e7-987109d40d67 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:18.763012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:18.791914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:18.792689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:18.792965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:18.793231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:18.794859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:18.892259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:18.893871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdf4fd-d39c-4f75-a7cc-7ac59cf7c6b3 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:18.912955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 724/1434] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:19.724339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:19.725018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:19.725018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:19.726248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:19.731884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215bbc3-7a78-4d20-9ad1-be23867115cf None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:19.732370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215bbc3-7a78-4d20-9ad1-be23867115cf None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:19.732470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3215bbc3-7a78-4d20-9ad1-be23867115cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:19.814522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3215bbc3-7a78-4d20-9ad1-be23867115cf None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:19.815779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3215bbc3-7a78-4d20-9ad1-be23867115cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:19.859173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:19.916404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:29:20.076305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:20.077785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:20.235457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224d1de-f48b-4826-8cde-fed8eb4281ba None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:20.289737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:20.352557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:20.353078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:20.353359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:20.353626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:20.355321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:20.450885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:20.452545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bd2b0e-b246-4b2e-be0b-67a92e3a65b2 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:20.471275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 712/1437] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:21.278519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:21.279167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:21.279167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:21.280459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:21.287958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a40-c114-43a1-8c75-8460efb75615 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:21.288454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a40-c114-43a1-8c75-8460efb75615 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:21.289187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b18a40-c114-43a1-8c75-8460efb75615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:21.403691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-07b18a40-c114-43a1-8c75-8460efb75615 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:21.404930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-07b18a40-c114-43a1-8c75-8460efb75615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:21.407041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:21.520154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:21.669835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:21.671410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:21.831555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442db9c-95ab-41ea-a20c-4209d6906ed9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:21.860615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:21.953583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:21.954194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:21.954469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:21.954732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:21.956377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:22.050202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:22.051913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fca0a-5dca-43da-807c-c7253e3190fb service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:22.071151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 727/1440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:24.077904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:24.078752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:24.078752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:24.080562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:24.086646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d409b-2494-4eac-b40c-d03313331279 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:24.087440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d409b-2494-4eac-b40c-d03313331279 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:24.087440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d409b-2494-4eac-b40c-d03313331279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:24.261615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:24.289059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-122d409b-2494-4eac-b40c-d03313331279 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:24.290680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-122d409b-2494-4eac-b40c-d03313331279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:24.396864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:24.436330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:24.438642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:24.572765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62db878-bd7c-49ae-9ffd-a2fdd5eb3093 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:24.609727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:24.669805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:24.670440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:24.670715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:24.670973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:24.672712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:24.796220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:24.797706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652f7c6-c25b-45ab-9768-9ff29e88465d service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:24.816456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 715/1443] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:26.986405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:26.986989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:26.986989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:26.988383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:26.993937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7351-55c1-4f62-ba1b-2f8af3d09822 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:26.994410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7351-55c1-4f62-ba1b-2f8af3d09822 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:26.994526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffa7351-55c1-4f62-ba1b-2f8af3d09822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:27.064128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-bffa7351-55c1-4f62-ba1b-2f8af3d09822 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:27.065360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bffa7351-55c1-4f62-ba1b-2f8af3d09822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:27.078499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:27.166758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:27.206154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:27.208599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:27.321873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b182ab7-9612-433d-b780-5af8a2be6bb8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:27.335510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:27.423616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:27.424227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:27.424496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:27.424763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:27.426428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:27.538016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:27.539512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bd65db-2c44-4e6f-a42b-f9f4fd69c83f service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:27.557594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 730/1446] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:29.963661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:29.964275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:29.964275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:29.965567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:29.971119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8857c-9f11-40cd-aa9f-f5193c3d76ae None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:29.971572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8857c-9f11-40cd-aa9f-f5193c3d76ae None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:29.971665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8857c-9f11-40cd-aa9f-f5193c3d76ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:30.022849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:30.048489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b4d8857c-9f11-40cd-aa9f-f5193c3d76ae None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:30.049729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b4d8857c-9f11-40cd-aa9f-f5193c3d76ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:30.158465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:30.198469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:30.200152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:30.311445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0be1dfb-2d5f-49c9-9e40-7135e7b68ebe None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:30.352591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:30.391528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:30.392444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:30.392444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:30.392691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:30.394357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:30.511266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:30.512736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90897594-4e5e-4b34-b716-004609f781a1 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:30.535845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 718/1449] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:32.678532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:32.679185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:32.679185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:32.680467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:32.686204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978869-f990-4ba7-a9bf-442eb5634bc6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:32.686664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978869-f990-4ba7-a9bf-442eb5634bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:32.686786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31978869-f990-4ba7-a9bf-442eb5634bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:32.753955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-31978869-f990-4ba7-a9bf-442eb5634bc6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:32.755202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-31978869-f990-4ba7-a9bf-442eb5634bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:32.776430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:32.852666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:32.892434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:32.894012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:33.003459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ff3e0-759b-46e1-a8e0-e9900097b100 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:33.023920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:33.080947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:33.081551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:33.082169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:33.082447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:33.084124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:33.203583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:33.205060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e63060-8134-452d-b449-31a7dab01d52 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:33.224380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 733/1452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:35.279936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:35.280800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:35.280800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:35.281887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:35.287924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e843-5457-4a63-9bee-f0cc01deb702 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:35.288479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e843-5457-4a63-9bee-f0cc01deb702 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:35.288479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1580e843-5457-4a63-9bee-f0cc01deb702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:35.357701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:35.361349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1580e843-5457-4a63-9bee-f0cc01deb702 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:35.362964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1580e843-5457-4a63-9bee-f0cc01deb702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:35.471235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:35.510793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:35.512420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:35.620664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b743b-32a3-4827-86cd-484d55319c6d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:35.670488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:35.697389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:35.698011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:35.698281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:35.698619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:35.700256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:35.817495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:35.818855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bff440-4e39-4b7a-8da9-98779cf18206 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:35.837719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 721/1455] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:37.958760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:37.959383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:37.959383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:37.960702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:37.968819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bba52-596e-427d-970e-58e61db86f2d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:37.969362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bba52-596e-427d-970e-58e61db86f2d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:37.969362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7bba52-596e-427d-970e-58e61db86f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:38.048904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8f7bba52-596e-427d-970e-58e61db86f2d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:38.050146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8f7bba52-596e-427d-970e-58e61db86f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:38.080266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:38.149906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:38.188757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:38.190362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:38.303805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d525b1-6d1e-46df-800e-5bab91b30157 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:38.333743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:38.381157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:38.381775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:38.382043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:38.382309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:38.383986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:38.495469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:38.497285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670bbad5-a710-49ad-a012-f6ba0c4e3e1d service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:38.516181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 736/1458] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:40.694342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:40.694953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:40.694953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:40.696470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:40.701970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4b91-b806-4511-95e8-007dca7338a3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:40.702458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4b91-b806-4511-95e8-007dca7338a3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:40.702458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c4b91-b806-4511-95e8-007dca7338a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:40.774429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-340c4b91-b806-4511-95e8-007dca7338a3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:40.775879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-340c4b91-b806-4511-95e8-007dca7338a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:40.784959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:40.879097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:40.919465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:40.921016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:41.035257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f4abb8-24d6-4a5e-884d-6d9c4e67ff77 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:41.093233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:41.112460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:41.113078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:41.113358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:41.113623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:41.115284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:41.229227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:41.230693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957269d6-c2d2-4a42-89e3-1aeeca4db427 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:41.255751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 724/1461] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:43.314731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:43.315379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:43.315379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:43.316765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:43.322956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f187c6-2ae1-4878-80b1-00d89a75eb2a None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:43.323583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f187c6-2ae1-4878-80b1-00d89a75eb2a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:43.323583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f187c6-2ae1-4878-80b1-00d89a75eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:43.396502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d6f187c6-2ae1-4878-80b1-00d89a75eb2a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:43.397744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d6f187c6-2ae1-4878-80b1-00d89a75eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:43.438214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:43.503545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:43.541510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:43.543060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:43.657623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a18e13b-08e1-434a-9bcd-523757b07e18 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:43.696365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:43.734578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:43.735213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:43.735492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:43.735755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:43.737352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:43.848221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:43.849583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e577da-e240-4d9a-b7cc-b86b1b3d5897 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:43.867830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 739/1464] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:45.913069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:45.913732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:45.913732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:45.914993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:45.921328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ada9-916b-4188-9248-5c087af9c44c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:45.921980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ada9-916b-4188-9248-5c087af9c44c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:45.921980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439ada9-916b-4188-9248-5c087af9c44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:45.990969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6439ada9-916b-4188-9248-5c087af9c44c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:45.992240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6439ada9-916b-4188-9248-5c087af9c44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:46.011363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:46.092744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:29:46.129882 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:46.131453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:46.242779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b11d5d-8f99-4e17-a8ea-ef57fbae3850 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:46.313571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:46.321565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:46.322283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:46.322554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:46.322845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:46.324492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:46.440311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:46.441741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5323d65-81c8-4ed8-b6ad-ecdad4945122 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:46.460526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 727/1467] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:48.487127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:48.487738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:48.487738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:48.489062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:29:48.495434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec4ad-7960-46d1-9c8f-424ebdfaf0dc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:48.496441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec4ad-7960-46d1-9c8f-424ebdfaf0dc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:48.496441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fec4ad-7960-46d1-9c8f-424ebdfaf0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:48.566653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c5fec4ad-7960-46d1-9c8f-424ebdfaf0dc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:29:48.567811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c5fec4ad-7960-46d1-9c8f-424ebdfaf0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:48.620418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:48.670554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:29:48.707504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:29:48.709097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:48.821816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e67f6-262a-4879-803e-fab16cce4b50 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:29:48.873183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:48.901249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:29:48.901858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:29:48.902136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:29:48.902408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:29:48.904058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:29:49.018461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:29:49.019967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f194281-8bd1-4d98-aba8-04c1043c6c31 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:29:49.040187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 742/1470] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:29:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:29:58.668697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=22 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:29:59.037541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=66 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:08.512734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85951a1-b00c-4147-b5ce-a8254330a167 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:08.513337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85951a1-b00c-4147-b5ce-a8254330a167 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:08.513337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85951a1-b00c-4147-b5ce-a8254330a167 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:08.514625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 729/1471] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:08.520847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c353c-f1e1-4bdd-86ac-28a36fca7941 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:08.521444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c353c-f1e1-4bdd-86ac-28a36fca7941 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:08.521444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c353c-f1e1-4bdd-86ac-28a36fca7941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:08.593659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-627c353c-f1e1-4bdd-86ac-28a36fca7941 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:08.594960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-627c353c-f1e1-4bdd-86ac-28a36fca7941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:08.704061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 743/1472] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:30:08.791104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:08.791458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:08.791734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:08.793156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 730/1473] 10.222.0.33 () {54 vars in 967 bytes} [Mon Aug 8 13:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:08.798307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00a818-eea9-4908-af4b-98b1d40d6d55 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:08.798594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00a818-eea9-4908-af4b-98b1d40d6d55 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:08.798867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00a818-eea9-4908-af4b-98b1d40d6d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:08.880245 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2e00a818-eea9-4908-af4b-98b1d40d6d55 None None] MFA Rules not processed for user `f034fd79c9ef454498ee0874d2c97d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:08.881486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2e00a818-eea9-4908-af4b-98b1d40d6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:08.903597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:08.963273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 744/1474] 10.222.0.33 () {58 vars in 1064 bytes} [Mon Aug 8 13:30:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:08.970974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:08.972583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:09.079408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608988a-b42a-4db3-96ea-8793f43a0ee3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:09.156245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:09.156862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:09.157143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:09.157414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:09.159069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:09.217572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:09.276654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:09.278116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92160aa1-db40-4c3a-8f9b-279e6b180168 service glance] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:09.297361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 731/1475] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:11.457797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:11.458351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:11.458351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:11.459680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 745/1476] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:11.465055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b176-dfc7-48cd-a584-6ac91b8d454d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:11.465541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b176-dfc7-48cd-a584-6ac91b8d454d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:11.465541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6b176-dfc7-48cd-a584-6ac91b8d454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:11.534936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-cde6b176-dfc7-48cd-a584-6ac91b8d454d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:11.536207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-cde6b176-dfc7-48cd-a584-6ac91b8d454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:11.649702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 732/1477] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:11.673182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:11.674931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:11.807059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17a8ea-f867-4b6d-aef3-8f2555c6dd5c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:11.855106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:11.901614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:11.902225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:11.902494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:11.902731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:11.903426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:11.905184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:30:11.905184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:30:11.905184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:30:11.906312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149deacf-64e1-4a00-8913-8ec8cdfa5fdf admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:11.925297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 746/1478] 10.222.0.33 () {56 vars in 1108 bytes} [Mon Aug 8 13:30:11 2022] GET /identity/v3/projects => generated 2553 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:30:13.993931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:13.994588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:13.994588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:13.995884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 733/1479] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:14.001685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a65839-d221-4c73-98d7-4109ff502147 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:14.002205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a65839-d221-4c73-98d7-4109ff502147 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:14.002297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a65839-d221-4c73-98d7-4109ff502147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:14.074801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b9a65839-d221-4c73-98d7-4109ff502147 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:14.076100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b9a65839-d221-4c73-98d7-4109ff502147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:14.185273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 747/1480] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:14.230640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:14.232268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:14.242184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:14.391593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33956a1d-7c7e-49bc-823e-1ce38552661d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:14.475678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:14.513505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:14.514128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:14.514404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:14.514667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:14.516743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:14.625301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:14.626936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c375ba-d01f-45b4-8a3d-2879ad25cc50 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:14.645357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 734/1481] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:16.867386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:16.867965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:16.867965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:16.869410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 748/1482] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:16.875375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1cac-8361-4216-b122-f5f99f7d0767 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:16.875915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1cac-8361-4216-b122-f5f99f7d0767 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:16.876012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1cac-8361-4216-b122-f5f99f7d0767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:16.946502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fa8d1cac-8361-4216-b122-f5f99f7d0767 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:16.947792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fa8d1cac-8361-4216-b122-f5f99f7d0767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:17.005080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:17.057062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 735/1483] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:17.095150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:17.096910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:17.123161 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:17.216398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f25516-35f9-4fd1-bd20-5b7d50ee17e2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:17.293882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:17.294486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:17.294759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:17.295026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:17.296678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:17.380026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:17.419638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:17.421094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7261349-de79-45a0-8211-ceeee98988cd service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:17.439933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 749/1484] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:19.508539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:19.509107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:19.509107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:19.510524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 736/1485] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:19.516094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71298045-5617-4dab-8cc5-90e5a9ef5008 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:19.516601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71298045-5617-4dab-8cc5-90e5a9ef5008 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:19.516601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71298045-5617-4dab-8cc5-90e5a9ef5008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:19.588824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-71298045-5617-4dab-8cc5-90e5a9ef5008 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:19.590048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-71298045-5617-4dab-8cc5-90e5a9ef5008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:19.692606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 750/1486] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:19.732746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:19.734359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:19.852027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29391ba3-996c-4048-8182-77c9142709e9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:19.928026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:19.930086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:19.930895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:19.931301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:19.931697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:19.933506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:20.056220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:20.056220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03ff1d-73ac-40bd-8987-da6f45c89fee service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:20.075280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 737/1487] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:22.294079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:22.294794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:22.294794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:22.295997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 751/1488] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:22.301792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb88ab-e8fd-4d2f-9754-53bb773c1191 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:22.302286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb88ab-e8fd-4d2f-9754-53bb773c1191 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:22.302286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb88ab-e8fd-4d2f-9754-53bb773c1191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:22.372086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6deb88ab-e8fd-4d2f-9754-53bb773c1191 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:22.373313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6deb88ab-e8fd-4d2f-9754-53bb773c1191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:22.430839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:22.481186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 738/1489] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:22.522606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:22.524194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:22.564120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:22.637834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e7ecf1-52f1-4b20-971e-ba7656387f5b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:22.717040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:22.717705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:22.717988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:22.718267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:22.720018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:22.803569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:22.834341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:22.835772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273bb872-cdc0-4a81-aa7f-1476feed6228 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:22.854269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 752/1490] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:25.129759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:25.130336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:25.130336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:25.131728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 739/1491] 10.222.0.33 () {54 vars in 927 bytes} [Mon Aug 8 13:30:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:30:25.137374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd648-4c3e-4934-ae73-afbddb7682d4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:25.137865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd648-4c3e-4934-ae73-afbddb7682d4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:25.137964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014cd648-4c3e-4934-ae73-afbddb7682d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:25.207787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-014cd648-4c3e-4934-ae73-afbddb7682d4 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:30:25.209037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-014cd648-4c3e-4934-ae73-afbddb7682d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:25.318126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 753/1492] 10.222.0.33 () {58 vars in 1024 bytes} [Mon Aug 8 13:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:30:25.358157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:30:25.359826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:25.469746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0529a3b-edeb-4730-abda-ac4d4d8f6d91 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:30:25.544951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:30:25.545563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:30:25.545848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:30:25.546113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:30:25.547761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:30:25.557197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:25.668217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:30:25.669701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d3f08-9db8-4922-bf15-cb3023fb8f8d service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:30:25.690130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 740/1493] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:30:35.316166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=82 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:30:35.687601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=50 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:43.682162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:43.682780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:43.682780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:43.761896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:31:43.763114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:43.864422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 754/1494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:31:43.873647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:43.874225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:43.874225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:43.952777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:31:43.954005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:44.057491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 741/1495] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:31:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:31:44.066656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.067936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:44.079656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db38c59-374d-47a0-857f-ba9134a4032a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:44.147229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:44.187981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:44.188596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:44.188873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:44.189140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:44.189786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:31:44.190084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:44.191164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:44.191164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:31:44.191164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:31:44.192057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:44.204766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 755/1496] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:31:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:31:44.217779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.229850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64a611-9c78-41c4-b727-dc44fe9be4b7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:44.294728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:44.332358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:44.332968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:44.333242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:44.333508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:44.334131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:44.335941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:44.335941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:31:44.335941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:31:44.337131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:44.403189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 742/1497] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:31:44 2022] POST /identity/v3/projects => generated 379 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:31:44.408081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.409671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:44.449509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a32af-2b97-44bf-b354-d4101e2384dd admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:44.569174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:44.569691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:44.569965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:44.570252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:44.570876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:44.571968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:44.571968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:31:44.571968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:31:44.573216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:44.619585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 756/1498] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:31:44 2022] POST /identity/v3/users => generated 290 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:31:44.624334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.625887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:44.693082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6660123-e7fe-4880-9255-4aa3124e813c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:44.791478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:44.804866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:44.805520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:44.805790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:44.806056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:44.806723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:44.807853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:44.807853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:31:44.807853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:31:44.808774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:44.824756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 743/1499] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:31:44.829798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.831412 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:44.843444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ba7641-ed71-4e2b-8963-ad9bee5ff127 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:44.872257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:44.946864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:44.947545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:44.947822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:44.948086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:44.948730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:44.949776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:44.949776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:31:44.949776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:31:44.950673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c3100a-66b2-414b-b7ca-aa00a7959867 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:44.963741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 757/1500] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:31:44 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:31:44.968418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:44.969996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:45.007224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2666d5a-dafc-49ff-a85e-6505e4f7207c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:45.109742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:45.110325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.110598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.111794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] PATH_INFO: `/v3/projects/e487d40c05054eb1a7897d4968ed362e/users/39db3712977d437186860497c2f078db/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.142869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e487d40c05054eb1a7897d4968ed362e, user_id=39db3712977d437186860497c2f078db, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:45.150359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:45.150359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:31:45.150359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:31:45.152382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a204ccd7-521a-418b-8bb2-9117d2b12e16 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:45.187045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 744/1501] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:31:44 2022] PUT /identity/v3/projects/e487d40c05054eb1a7897d4968ed362e/users/39db3712977d437186860497c2f078db/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:31:45.201782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9b4b-85e6-4f62-a0c9-8e3e7c325ffa None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.202122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9b4b-85e6-4f62-a0c9-8e3e7c325ffa None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.202407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a9b4b-85e6-4f62-a0c9-8e3e7c325ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.269683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-250a9b4b-85e6-4f62-a0c9-8e3e7c325ffa None None] MFA Rules not processed for user `39db3712977d437186860497c2f078db`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:31:45.270929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-250a9b4b-85e6-4f62-a0c9-8e3e7c325ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:45.365407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 758/1502] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:31:45.379644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.379930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.380247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.381558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 745/1503] 10.222.0.33 () {54 vars in 964 bytes} [Mon Aug 8 13:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:31:45.387020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689e7a-5a56-4ec1-8c21-7d0a91affdd7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.387346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689e7a-5a56-4ec1-8c21-7d0a91affdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.387618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19689e7a-5a56-4ec1-8c21-7d0a91affdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.423159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:45.474044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-19689e7a-5a56-4ec1-8c21-7d0a91affdd7 None None] MFA Rules not processed for user `656cb98cf9b147f993f2cd31ec4ea55c`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:31:45.475322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-19689e7a-5a56-4ec1-8c21-7d0a91affdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:45.559975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 759/1504] 10.222.0.33 () {58 vars in 1061 bytes} [Mon Aug 8 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:31:45.568772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:45.570570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:45.632017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:45.688423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff5b2cc-8d28-4f9c-8bec-2c6504af3b4f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:45.775884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:45.776493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.776762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.777012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.778898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:45.887974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:45.889549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:45.894257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:45.910811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 746/1505] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:31:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:31:45.962996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bacf-8e49-4ba6-a16e-76232e418a5d None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:45.963478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bacf-8e49-4ba6-a16e-76232e418a5d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:45.963582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891bacf-8e49-4ba6-a16e-76232e418a5d None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:45.964876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 760/1506] 10.222.0.33 () {52 vars in 830 bytes} [Mon Aug 8 13:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:31:45.968835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:45.970066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:46.076097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b10fe-d55c-4351-aaa3-e03cbdbff3e3 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:46.179859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:46.180511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:46.180787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:46.181059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] PATH_INFO: `/v3/users/39db3712977d437186860497c2f078db` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:46.195289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39db3712977d437186860497c2f078db)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:46.196374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:46.196374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:31:46.196374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:31:46.197996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:46.212228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:46.276967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39db3712977d437186860497c2f078db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:46.293268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39db3712977d437186860497c2f078db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:46.304292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39db3712977d437186860497c2f078db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:46.304932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39db3712977d437186860497c2f078db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:46.305296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39db3712977d437186860497c2f078db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:46.334660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 747/1507] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:31:45 2022] DELETE /identity/v3/users/39db3712977d437186860497c2f078db => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:31:46.341066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:46.341346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:46.341616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:46.404726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:31:46.406070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:46.486491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:46.500053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 761/1508] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:31:46.510806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:46.512130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:46.629159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c80360-4d49-49ea-a557-40ec235fbfe3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:46.689313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:46.715550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:46.716184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:46.716477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:46.716740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:46.718545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:46.836411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:46.837886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dff1b5-b4d9-49fd-9987-6451b9c5daa9 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:46.856811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 748/1509] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:31:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:31:47.315838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:31:47.317987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:31:47.341872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1707c6-bef4-44b5-b090-4168cd3edc4d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:31:47.454625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:31:47.455281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:31:47.455562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:31:47.455834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] PATH_INFO: `/v3/projects/e487d40c05054eb1a7897d4968ed362e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:31:47.466575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e487d40c05054eb1a7897d4968ed362e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:31:47.467665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:31:47.467665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:31:47.467665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:31:47.468883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:31:47.541575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e487d40c05054eb1a7897d4968ed362e', 'request_id': 'req-c48844be-53a6-4f04-a2b9-b5019190a094'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:47.548060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e487d40c05054eb1a7897d4968ed362e', 'request_id': 'req-c48844be-53a6-4f04-a2b9-b5019190a094'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:47.559104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e487d40c05054eb1a7897d4968ed362e', 'request_id': 'req-c48844be-53a6-4f04-a2b9-b5019190a094'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:47.597376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] The token cache is being invalidate because project e487d40c05054eb1a7897d4968ed362e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:31:47.597672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:47.598217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c48844be-53a6-4f04-a2b9-b5019190a094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:31:47.599746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 762/1510] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:31:47 2022] DELETE /identity/v3/projects/e487d40c05054eb1a7897d4968ed362e => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:31:56.854429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=62 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:31:57.594907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:33:14.782777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:14.783473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:14.783473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:14.784715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:14.790654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbee0-6bda-4660-a982-7d153375cb4c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:14.791160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbee0-6bda-4660-a982-7d153375cb4c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:14.791276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfecbee0-6bda-4660-a982-7d153375cb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:14.882868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-bfecbee0-6bda-4660-a982-7d153375cb4c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:14.884137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bfecbee0-6bda-4660-a982-7d153375cb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:15.002005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:33:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:15.159313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:15.161510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:15.286309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff94c37-3931-4e2c-a67e-a67128a1f620 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:15.292012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:15.292617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:15.292890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:15.293158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:15.301739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:15.302764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9428c4-34e6-4352-b9ae-47e6ea852a0a service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:15.312832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 750/1513] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:16.190201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8026-e78a-4629-8504-a07a0ba166d8 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:16.191155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8026-e78a-4629-8504-a07a0ba166d8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:16.191155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8026-e78a-4629-8504-a07a0ba166d8 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:16.193288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:33:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:16.199660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37c52-b4b8-4d4a-adcf-001d891ea8bb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:16.200077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37c52-b4b8-4d4a-adcf-001d891ea8bb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:16.200235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37c52-b4b8-4d4a-adcf-001d891ea8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:16.221297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1ee37c52-b4b8-4d4a-adcf-001d891ea8bb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:16.222514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1ee37c52-b4b8-4d4a-adcf-001d891ea8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:16.232140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:16.387415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8026-e78a-4629-8504-a07a0ba166d8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:16.406862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8026-e78a-4629-8504-a07a0ba166d8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:16.412465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:16.413069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:16.413362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:16.413617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:16.422508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:16.424163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a09b01-88eb-46ce-bb7b-c6bbfbdee3b6 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:16.434106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 765/1516] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:16.543957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c309c7-ba0d-46c0-afb4-b72f2a8c01ca None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:16.544305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c309c7-ba0d-46c0-afb4-b72f2a8c01ca None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:16.544675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c309c7-ba0d-46c0-afb4-b72f2a8c01ca None None] PATH_INFO: `/v3` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:16.546047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 752/1517] 10.222.0.48 () {54 vars in 925 bytes} [Mon Aug 8 13:33:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:33:16.553825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5eb1-513c-4da6-9e3a-4a1cdb5a54f4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:16.554133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5eb1-513c-4da6-9e3a-4a1cdb5a54f4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:16.554505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543d5eb1-513c-4da6-9e3a-4a1cdb5a54f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:16.620016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-543d5eb1-513c-4da6-9e3a-4a1cdb5a54f4 None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:16.621257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-543d5eb1-513c-4da6-9e3a-4a1cdb5a54f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:16.638802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:33:16.699099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 766/1518] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 8 13:33:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:25.627054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3d4aa-09d7-468a-bc5a-7167b83ad7cb None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:25.627678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3d4aa-09d7-468a-bc5a-7167b83ad7cb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:25.627678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc3d4aa-09d7-468a-bc5a-7167b83ad7cb None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:25.628872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 753/1519] 10.222.0.33 () {54 vars in 961 bytes} [Mon Aug 8 13:33:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:25.634095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398812-3e53-4100-bc8b-c133f59fe87d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:25.634609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398812-3e53-4100-bc8b-c133f59fe87d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:25.634717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f398812-3e53-4100-bc8b-c133f59fe87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:25.655190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1f398812-3e53-4100-bc8b-c133f59fe87d None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:25.656410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1f398812-3e53-4100-bc8b-c133f59fe87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:25.666838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 767/1520] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Aug 8 13:33:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:25.674783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3d4aa-09d7-468a-bc5a-7167b83ad7cb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:25.688859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc3d4aa-09d7-468a-bc5a-7167b83ad7cb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:25.693949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:25.694560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:25.694841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:25.695140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:25.704336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:25.705986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944ca9db-6e45-456d-bebc-9d664269fb69 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:25.716719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 754/1521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:27.664752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd2ad-7008-4f58-9375-8b77976bf61c None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:27.665656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd2ad-7008-4f58-9375-8b77976bf61c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:27.665656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7fd2ad-7008-4f58-9375-8b77976bf61c None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:27.667871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:33:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:27.672909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec2269-db12-444c-a9ae-fbe5853e1f30 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:27.673376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec2269-db12-444c-a9ae-fbe5853e1f30 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:27.673481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec2269-db12-444c-a9ae-fbe5853e1f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:27.695267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9bec2269-db12-444c-a9ae-fbe5853e1f30 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:27.696515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9bec2269-db12-444c-a9ae-fbe5853e1f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:27.705529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:27.858757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fd2ad-7008-4f58-9375-8b77976bf61c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:27.882378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7fd2ad-7008-4f58-9375-8b77976bf61c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:27.889695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:27.890281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:27.890581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:27.890877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:27.899499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:27.901114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c2dc7d-a82b-428b-a61a-96d9ebb0a2d3 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:27.910945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 769/1524] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:28.746813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca92f0-d501-477e-91c6-ee93c95f9ff3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:28.747766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca92f0-d501-477e-91c6-ee93c95f9ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:28.747766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cca92f0-d501-477e-91c6-ee93c95f9ff3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:28.749845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Aug 8 13:33:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:28.755947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf47d1-1b6c-4f60-be6c-2241c4117b05 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:28.756427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf47d1-1b6c-4f60-be6c-2241c4117b05 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:28.756544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcf47d1-1b6c-4f60-be6c-2241c4117b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:28.777260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4dcf47d1-1b6c-4f60-be6c-2241c4117b05 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:28.781044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4dcf47d1-1b6c-4f60-be6c-2241c4117b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:28.789776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Aug 8 13:33:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:28.944065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca92f0-d501-477e-91c6-ee93c95f9ff3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:28.966028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cca92f0-d501-477e-91c6-ee93c95f9ff3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:28.972325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:28.972929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:28.973210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:28.973518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:28.982203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:28.983844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1131b524-0cd7-4ea3-9a04-c4f47a836f08 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:28.993340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 757/1527] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:32.763899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:32.764473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:32.764473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:32.765803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:32.772070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7aa56-d28d-4906-87f0-997c6389bca2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:32.772527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7aa56-d28d-4906-87f0-997c6389bca2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:32.772640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7aa56-d28d-4906-87f0-997c6389bca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:32.794811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-55d7aa56-d28d-4906-87f0-997c6389bca2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:32.795971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-55d7aa56-d28d-4906-87f0-997c6389bca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:32.804982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:32.896496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:32.899735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:32.982319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491240fc-a885-45a6-910b-b9b14ea81778 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:32.986355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:32.986955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:32.987259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:32.987528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:32.997040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:32.998468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b027b-33c1-40e9-a605-abc87423a78f service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:33.008364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 772/1530] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:35.014989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:35.015605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:35.015605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:35.017060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:33:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:33:35.023021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d154-5c76-4f26-b19a-49f03dfa7d67 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:35.023506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d154-5c76-4f26-b19a-49f03dfa7d67 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:35.023600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad72d154-5c76-4f26-b19a-49f03dfa7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:35.044479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ad72d154-5c76-4f26-b19a-49f03dfa7d67 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:35.045734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ad72d154-5c76-4f26-b19a-49f03dfa7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:35.054678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:33:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:35.092269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:35.094175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:35.112878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c79f8-2e75-4c6d-b513-876dee9718e3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:35.119595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:35.120226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:35.120496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:35.120763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:35.131540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:35.132998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ea6b1-596f-4d4c-98ca-d6c388a1c121 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:35.143211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 760/1533] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:38.616281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:38.616838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:38.616838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:38.618713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:33:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:38.626984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d7b-2ccc-4c7e-bad8-99fe2e42e8ee None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:38.627458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d7b-2ccc-4c7e-bad8-99fe2e42e8ee None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:38.627588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da50d7b-2ccc-4c7e-bad8-99fe2e42e8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:38.639153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=81 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:33:38.651950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3da50d7b-2ccc-4c7e-bad8-99fe2e42e8ee None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:38.653138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3da50d7b-2ccc-4c7e-bad8-99fe2e42e8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:38.661505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:33:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:38.966314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:38.968113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:39.040642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:33:39.057816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58aa75-7b46-4c61-85ee-ad6e2bba97a5 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:39.062031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:39.062634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:39.062908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:39.063201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:39.072801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:39.074307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4badf2fd-dc35-42f5-9814-115599cca2c9 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:39.083747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 775/1536] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:33:41.074546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:41.075185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:41.075185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:41.076469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Aug 8 13:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:33:41.082421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb9472-281a-4a13-9bbf-759ec85b9d8d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:41.083007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb9472-281a-4a13-9bbf-759ec85b9d8d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:41.083007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb9472-281a-4a13-9bbf-759ec85b9d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:41.103885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c6fb9472-281a-4a13-9bbf-759ec85b9d8d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:33:41.105097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c6fb9472-281a-4a13-9bbf-759ec85b9d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:41.114119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Aug 8 13:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:33:41.419185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:33:41.421291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:33:41.436854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63b8926-4fe0-4a3e-a65f-f135a7634d0e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:33:41.440757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:33:41.441396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:33:41.441674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:33:41.441937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:33:41.451615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:33:41.453054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66db7d4-c0a2-4e06-a8f8-4a769cf5e48a service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:33:41.462854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 763/1539] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:33:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:33:51.098811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=11 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:33:51.458649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=8 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:05.509992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.511151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.511658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.540213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:05.541885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:05.555489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 777/1540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:05.565515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.566222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.566222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.593027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:05.594274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:05.611274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 764/1541] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:05.617152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:05.633780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a34e00-a24e-4419-8681-326682998ec0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:05.643222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:05.644532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.645153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.645676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.646654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:05.647247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:05.649489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6975ca7e-c2f9-4003-b880-9c7e9288663e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:05.671378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 778/1542] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:05.680523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:05.693726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e455b7-a89f-450c-8dce-1c6c28380b7c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:05.702496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:05.703183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.703465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.703740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.704399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:05.706481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:05.750887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.751630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.752134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.769886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 765/1543] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/projects => generated 377 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:05.774854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:05.792219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:05.793975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:05.850735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13212e-70ee-46fc-977d-807b3c5b8fcd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:05.855968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:05.856630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.857088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.857372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.858071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:05.859148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:05.859148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:35:05.859148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:35:05.860710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59daef4-7f45-4b7b-a7c6-227c925e1b6b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:05.889232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 766/1544] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:05.890677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fda0-2528-4c9b-8dbf-b6675d368a2c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.890975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fda0-2528-4c9b-8dbf-b6675d368a2c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.895486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a1fda0-2528-4c9b-8dbf-b6675d368a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.919188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 779/1545] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:05.920512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:05.931103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a2a1fda0-2528-4c9b-8dbf-b6675d368a2c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:05.931103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a2a1fda0-2528-4c9b-8dbf-b6675d368a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:05.939117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f792c7-3612-4b6e-b921-25ac9a1f9e63 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:05.945179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 767/1546] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:05.947155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e288-2235-4f9b-b060-3e61fae958bc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.947681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e288-2235-4f9b-b060-3e61fae958bc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.948170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9e288-2235-4f9b-b060-3e61fae958bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.951838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:05.959102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.959102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.959102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.959102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:05.959102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb25a34-eb95-48b9-9e77-efb0883caa6f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:05.973843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 780/1547] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:05.984872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dc05-332d-4a62-b68e-7ddce7033f42 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:05.986993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dc05-332d-4a62-b68e-7ddce7033f42 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:05.986993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0077dc05-332d-4a62-b68e-7ddce7033f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:05.994937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-29b9e288-2235-4f9b-b060-3e61fae958bc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:05.997819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-29b9e288-2235-4f9b-b060-3e61fae958bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.023224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0077dc05-332d-4a62-b68e-7ddce7033f42 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.024710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0077dc05-332d-4a62-b68e-7ddce7033f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.025348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 768/1548] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.026777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.027058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.027353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.034638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 781/1549] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.036383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164561-5bfd-4c7c-9288-f4e038672a73 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.036817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164561-5bfd-4c7c-9288-f4e038672a73 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.037401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d164561-5bfd-4c7c-9288-f4e038672a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.050060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.052340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.061496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 769/1550] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.062051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.066586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1d164561-5bfd-4c7c-9288-f4e038672a73 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.068249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1d164561-5bfd-4c7c-9288-f4e038672a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.073700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220d935f-304a-46ed-9fa6-3e4bb1009d54 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.077896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 782/1551] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.080326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef534-493d-4747-a086-11c7f55231e8 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.080466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.080676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.080949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.081223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.081467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef534-493d-4747-a086-11c7f55231e8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.081881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bef534-493d-4747-a086-11c7f55231e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.083799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.085545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.097564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 770/1552] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:06.098413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.102742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-55bef534-493d-4747-a086-11c7f55231e8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.103985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-55bef534-493d-4747-a086-11c7f55231e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.111757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e89536-af2d-4064-8ff9-37b39ca6f1d4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.112731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 783/1553] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.113772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f0aa7-48fc-4dd3-85c9-abea1e72afb5 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.114048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f0aa7-48fc-4dd3-85c9-abea1e72afb5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.114446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f0aa7-48fc-4dd3-85c9-abea1e72afb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.122867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.123532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.123807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.124076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.124731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.125058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.126266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:06.126266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:35:06.126266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:35:06.127179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d25aefa-669d-4429-b52d-d943e3fe2c73 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.135387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f36f0aa7-48fc-4dd3-85c9-abea1e72afb5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.137049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f36f0aa7-48fc-4dd3-85c9-abea1e72afb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.139714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 771/1554] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.140888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.141171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.141439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.151123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 784/1555] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.152378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27c779-1b4e-450d-86d6-c4c0f421c7f0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.152660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27c779-1b4e-450d-86d6-c4c0f421c7f0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.152930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af27c779-1b4e-450d-86d6-c4c0f421c7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.167423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.168628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.172762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-af27c779-1b4e-450d-86d6-c4c0f421c7f0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.174159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-af27c779-1b4e-450d-86d6-c4c0f421c7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.176925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 772/1556] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.177865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.186761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 785/1557] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.186761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129b5b-416e-4575-b9c2-f1b7e8f31cb3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.186761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129b5b-416e-4575-b9c2-f1b7e8f31cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.186761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129b5b-416e-4575-b9c2-f1b7e8f31cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.187780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e74f66-b913-4ccd-91f5-664f77d37659 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.192555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.193143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.193607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.193730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.194370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.194370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.196282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.206424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-21129b5b-416e-4575-b9c2-f1b7e8f31cb3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.207682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-21129b5b-416e-4575-b9c2-f1b7e8f31cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.210142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 773/1558] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.211024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.217365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 786/1559] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.219559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.219956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.220241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.222512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdd43b0-8c28-4ed7-afac-01b29be5fce9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.228621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.229295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.229572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.229844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] PATH_INFO: `/v3/projects/5021d13535274fa2b44666c8d39080bf/users/c93191a5e3c7476f926b56424ccec4ba/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.240496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.241673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.245903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5021d13535274fa2b44666c8d39080bf, user_id=c93191a5e3c7476f926b56424ccec4ba, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.248746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.249795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 787/1560] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.253101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.264170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9801e8-94e5-4770-b38b-7fffb3adb72a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.265646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 774/1561] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:06 2022] PUT /identity/v3/projects/5021d13535274fa2b44666c8d39080bf/users/c93191a5e3c7476f926b56424ccec4ba/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:06.266607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.297575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:06.324716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.325354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.325628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.325894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.326524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.327591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:06.327591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:35:06.327591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:35:06.328945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.334947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d155306c-f287-4c85-bcab-42cb11160d69 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.339831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.340463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.340719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.340986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.341669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.341969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.343619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb696e58-356c-4e6e-9c42-f273c956a54a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.356512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 775/1562] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.357775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.358085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.358366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.368005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 788/1563] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 383 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.369701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.383311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.384458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.385044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:06.440711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ec916-c9c2-40d2-9cfb-cd72245c4910 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.445597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 776/1564] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.445752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.446264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.446537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.446728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.446831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.447517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.447836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.449560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c78bf-cd76-44fd-adbe-6b01c07d4f94 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.457216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de99f2e-7534-41cf-b5bb-17058c83c8e1 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.462131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 789/1565] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.462695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.463408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.463532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.463690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.463795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.463964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.464087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.464599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.466581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afecf5b-6b7d-4f91-aa4e-a1a8c2f7de92 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.484551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.485847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.496713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 790/1566] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.497873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.500571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 777/1567] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.501858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.502129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.502834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.509637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c8d44-651c-4bad-9dcc-329df1afdbb3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.521534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:06.522758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:06.568550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.569149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.569440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.569802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.570464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.570799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.572475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.583297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 778/1568] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:06.584783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.595589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 791/1569] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.595589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.595911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04374c1e-34df-4db3-a080-232c843fd194 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.601222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.602015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.602288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.603065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.603238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.605112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.607046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed8268b-817d-485d-9f39-2d28583e6721 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.612041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.612801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.613106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.613379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.614012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.616099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.635850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 779/1570] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.640449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.641542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 792/1571] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.642249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.652164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b67a06-2d86-4da9-bef0-83a3814110f9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.658380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:06.704418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418b580-f873-481d-9653-fabaaff1ead7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.709495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.710120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.710399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.710664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.711353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.711657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.712509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.713126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.713286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.713418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.713667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.714295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.716264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.725247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 780/1572] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.726089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.736188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56292db8-7e7d-46c1-84a1-6abd7865949a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.741700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.742713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.743003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.743555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.744122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:06.744296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.745950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.746283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:06.758752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 793/1573] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 375 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.758752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.760446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 781/1574] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:06.761215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.772820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae12072-d7da-49dc-9da4-69da4fba47e0 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.822563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7093effd-7121-44d1-b543-369529c582e2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.827106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.827725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.828002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.828272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.828900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.830863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.831175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.831772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.832057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.832355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.832991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.834944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.854985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 794/1575] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.856172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.872809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 782/1576] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.872809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.917128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f606aff-12b0-48b5-a649-5f8cd6052946 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.931976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f08331-4ce4-4db3-90d1-e3a304546ead admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.936638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.937255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.937535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.937808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.938439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.940465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.962627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 783/1577] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:06.963179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.970876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.971561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.971842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.972111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.973187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae3aa36-6537-4194-be29-e79ad47aac02 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:06.977743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:06.978349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:06.978630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:06.978901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:06.979587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.981214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.982497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:06.984072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:06.987407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:06.993086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 784/1578] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:06.993923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:06.999536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 795/1579] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:07.000384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.005107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f7744-c035-4b93-86b8-b9445f08cbd1 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.010371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91147ae0-d811-43c8-a3a6-b9f6baf119b0 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.010574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.011225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.011571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.011781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.012405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.014554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.016605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.017255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.017935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.017935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.018536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.020554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.048525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 796/1580] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.048525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.053217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 785/1581] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:06 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.058100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.059771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0658cdc2-7a7a-45d4-9c77-5840832f275b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.090870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:07.130491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.131390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.131776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.132087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.133434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.135761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.136252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b645bd-c9a4-41c5-91ca-e6004128b2ac admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.141691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.142005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.142259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.142527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.143248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.144952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.158565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 786/1582] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.158565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.172840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9d5a7-fffe-4586-a95c-3e2d15a8aee3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.173099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 797/1583] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:07 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.173514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.247196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.250509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.250509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.250509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.250509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.251144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.256339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ffad09-df31-4ffd-81a8-6810e682519d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.265355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.266359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.266822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.267175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.268106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.272347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.278249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 787/1584] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:07 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.279576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.290977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 798/1585] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.290977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.295529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620f7376-54ba-4dc4-8e44-46834e7b7c33 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.301440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460d1a41-6bde-42ee-ae99-0f9b3fef8395 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.302419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.303070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.303421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.303710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.304414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.306149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.308209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.309017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.309340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.309712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.310479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.312636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.318934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 788/1586] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.320012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.328350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 799/1587] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.330043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05729171-671c-4b21-a91f-a95d055029a4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.332006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.335373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.336112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.336484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.336861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.337614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.339706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.342600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f29ae-0080-4fdc-ab39-a5b9a2707614 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.351110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.352473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.352833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.353064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.353785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.361360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.383780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 789/1588] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:07 2022] POST /identity/v3/users => generated 295 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.384911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.389514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 800/1589] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.390491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.400731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13627ca9-e0c5-41b5-b305-57564a092c92 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.401499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5810b5b-925b-4dce-9430-33384808019d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.406481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.406703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.407384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.407505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.407734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.407939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.408066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.408387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.408613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.409348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.411176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.411429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.423326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 801/1590] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.424442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.437692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0612b7-4d8b-42c2-977f-2a70ee80b9b7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.443197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.443991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.444670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.445050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.445872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.446518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 790/1591] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:07 2022] POST /identity/v3/users => generated 293 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:07.447772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.450999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.459411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775adee-5259-4d53-b8ef-13c7289f9c9e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.464937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 802/1592] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.464937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.468498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.469197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.469457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.469725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] PATH_INFO: `/v3/projects/4293ff5f2cad4c399d03a89f4bc627f5/users/fce6a7d7dbc94adeb0b03bae34e335a0/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.476854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2619966-5b0d-43ce-b6ab-abde4f52b1d8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.483046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.483788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.484399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.484673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.485606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.487296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.488944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4293ff5f2cad4c399d03a89f4bc627f5, user_id=fce6a7d7dbc94adeb0b03bae34e335a0, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.493011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.496300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:07.501020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 803/1593] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.501959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.513747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409588a3-bc34-40ad-b1d3-bdde4b962b6a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.515799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 791/1594] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:07 2022] PUT /identity/v3/projects/4293ff5f2cad4c399d03a89f4bc627f5/users/fce6a7d7dbc94adeb0b03bae34e335a0/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:07.521782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.571292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:07.588320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.589293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.589639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.590025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.591999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.592835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.593999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f918766-83c9-48a9-9a38-915c7001d3a4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.599066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.600238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.600238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.600522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] PATH_INFO: `/v3/projects/a3935a3e3fce40d7a936c74f38002fec/users/fa4870e52d734e64a2e07f82a4b59c71/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.606740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 804/1595] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.607754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.617855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3935a3e3fce40d7a936c74f38002fec, user_id=fa4870e52d734e64a2e07f82a4b59c71, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.619288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa79fd4-cc0b-4ce3-8647-d5141326fceb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.620646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.624405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.625042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.625338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.625698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.626264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.628474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.638703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 792/1596] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:07 2022] PUT /identity/v3/projects/a3935a3e3fce40d7a936c74f38002fec/users/fa4870e52d734e64a2e07f82a4b59c71/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:07.643724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 805/1597] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.643724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.648338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.662005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2899c9-d68f-4ffb-9476-189b687cf6b4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.720355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a1dcd5-b84c-43a5-b3f7-8ed3394e3618 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.726968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.727354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.727673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.727979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.729007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.730943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.731766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.732555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.732942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.733407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] PATH_INFO: `/v3/projects/f90db8c721374e0fb3a037661b3d9fe0/users/518d6948777944fd8214dc6d778a9a02/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.743280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 793/1598] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:07.744705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.750700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90db8c721374e0fb3a037661b3d9fe0, user_id=518d6948777944fd8214dc6d778a9a02, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.751732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:07.751732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:35:07.751732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:35:07.753914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40845e6-3b70-445e-b922-c8e004653e24 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.760985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b850db7-c694-459d-9fcc-ff7b746bb167 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.769484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 806/1599] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:07 2022] PUT /identity/v3/projects/f90db8c721374e0fb3a037661b3d9fe0/users/518d6948777944fd8214dc6d778a9a02/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:07.770157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.770443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.770872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.792607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:07.794048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:07.835786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.836476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.836804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.837081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] PATH_INFO: `/v3/projects/643240773a5840d3af2622469d696beb/users/651659e941044454985b2f866b791b07/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.857790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643240773a5840d3af2622469d696beb, user_id=651659e941044454985b2f866b791b07, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.861062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:07.865491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:07.885592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 794/1600] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:07 2022] PUT /identity/v3/projects/643240773a5840d3af2622469d696beb/users/651659e941044454985b2f866b791b07/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:07.886362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.910555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 807/1601] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:07.910555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:07.919203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ee0339-6a52-42e8-a64e-332ad6782380 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.984747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863ee7eb-ffd9-435c-8499-e9c32eba7d10 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:07.992680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:07.993470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:07.993761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:07.994040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:07.995066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:07.997438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5067988f-fcdc-4430-9514-d76c5d0152d5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.008891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.009860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.010239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.010609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] PATH_INFO: `/v3/projects/0367800b12d7428b9a58cff37f5fc501/users/a6ef5f1cd8e64a86bc691aee50fc699d/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.016175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 795/1602] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:08.017039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010bd-9dd6-4c5a-947f-e8ced3b4ae14 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.017678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010bd-9dd6-4c5a-947f-e8ced3b4ae14 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.018124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9010bd-9dd6-4c5a-947f-e8ced3b4ae14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.022080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:08.040493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0367800b12d7428b9a58cff37f5fc501, user_id=a6ef5f1cd8e64a86bc691aee50fc699d, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.044790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.059286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ef9010bd-9dd6-4c5a-947f-e8ced3b4ae14 None None] MFA Rules not processed for user `fa4870e52d734e64a2e07f82a4b59c71`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.061202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ef9010bd-9dd6-4c5a-947f-e8ced3b4ae14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.065049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 808/1603] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:07 2022] PUT /identity/v3/projects/0367800b12d7428b9a58cff37f5fc501/users/a6ef5f1cd8e64a86bc691aee50fc699d/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:08.066424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.155243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 796/1604] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.156708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e90b3-b550-4853-a67d-dca779fd0806 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.156994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e90b3-b550-4853-a67d-dca779fd0806 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.159442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875e90b3-b550-4853-a67d-dca779fd0806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.185162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-875e90b3-b550-4853-a67d-dca779fd0806 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.186677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-875e90b3-b550-4853-a67d-dca779fd0806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.188559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5a85f-c2cd-409f-961f-775bc0d4c15f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.195841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 797/1605] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.197523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.197976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.198410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.212524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.213569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.214740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.216210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.217726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.223183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.224507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.225947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.242838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 798/1606] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.242838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.260329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 809/1607] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:08.261786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.273727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314d153-9d59-411b-a38c-99449cbc768a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.279139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.280117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.280691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.281150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] PATH_INFO: `/v3/projects/0831512491044565b17fb6dd11a57b5f/users/c29ce05ebc6e4672bb9557f751a48557/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.299491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0831512491044565b17fb6dd11a57b5f, user_id=c29ce05ebc6e4672bb9557f751a48557, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.302658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d6427f-17a8-426e-8cf6-ccb065ce34b5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.321398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13df41c-df85-4d04-a70e-8e035770fc16 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.323354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 810/1608] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:08 2022] PUT /identity/v3/projects/0831512491044565b17fb6dd11a57b5f/users/c29ce05ebc6e4672bb9557f751a48557/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:08.325103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.325566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.326288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.372901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] MFA Rules not processed for user `a6ef5f1cd8e64a86bc691aee50fc699d`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.374146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.393185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:08.442801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.443834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.444305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.444864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.460620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 811/1609] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.466583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.478639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeb349-2c7f-46a1-9885-92a46ccad054 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.482856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.483909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.484275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.484473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.538528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.540515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a6a57-eee8-425b-9f15-51879a0c1d17 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.549906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:08.555536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 799/1610] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:08.557859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bde1a-870d-4e30-91b0-db6a6f637adb None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.558374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bde1a-870d-4e30-91b0-db6a6f637adb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.558820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3bde1a-870d-4e30-91b0-db6a6f637adb None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.560352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 800/1611] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 13:35:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:08.561425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bde1a-870d-4e30-91b0-db6a6f637adb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.584461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3bde1a-870d-4e30-91b0-db6a6f637adb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.598337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.599198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.599884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.600167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.600344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.600451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.620219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 812/1612] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:08.621346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:08.624469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.626017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafc65d8-ff3b-4c9f-bacc-c078a6a32d55 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.639712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 801/1613] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:08.641919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecb174-f483-4f0c-b1d1-66738aa7d7be None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.642210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecb174-f483-4f0c-b1d1-66738aa7d7be None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.642483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecb174-f483-4f0c-b1d1-66738aa7d7be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.645413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fe61c-2051-4e27-ab96-52b49d8f5d2d service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:08.663914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:08.664617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.664905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.665193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] PATH_INFO: `/v3/projects/7443e956a2d7470f878fcb96a99e6e40/users/f1a885a2de8e4af4b7ec02a52458ebc0/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.683184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a3ecb174-f483-4f0c-b1d1-66738aa7d7be None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.683390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7443e956a2d7470f878fcb96a99e6e40, user_id=f1a885a2de8e4af4b7ec02a52458ebc0, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:08.684792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a3ecb174-f483-4f0c-b1d1-66738aa7d7be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.685912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8ea3db-dad9-4caa-a3c0-503e94c6dd73 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:08.696287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 802/1614] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.698777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca09d1-f6f9-43ed-ac99-117e6e6814f4 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.699404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca09d1-f6f9-43ed-ac99-117e6e6814f4 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.699808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca09d1-f6f9-43ed-ac99-117e6e6814f4 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.707936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 803/1616] 10.222.0.33 () {54 vars in 962 bytes} [Mon Aug 8 13:35:08 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:08.709749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552278f-cacf-4255-97e5-720498287d12 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.710373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552278f-cacf-4255-97e5-720498287d12 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.710798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552278f-cacf-4255-97e5-720498287d12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.719193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 813/1616] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:08 2022] PUT /identity/v3/projects/7443e956a2d7470f878fcb96a99e6e40/users/f1a885a2de8e4af4b7ec02a52458ebc0/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:08.720689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1447-ae4d-4153-b6a0-9dfaccdcc337 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.720978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1447-ae4d-4153-b6a0-9dfaccdcc337 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.721275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6b1447-ae4d-4153-b6a0-9dfaccdcc337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.767210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3b6b1447-ae4d-4153-b6a0-9dfaccdcc337 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.767917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3b6b1447-ae4d-4153-b6a0-9dfaccdcc337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.791158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e552278f-cacf-4255-97e5-720498287d12 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.794649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e552278f-cacf-4255-97e5-720498287d12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.862330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 814/1617] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.864066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:08.864481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:08.864668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:08.902851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] MFA Rules not processed for user `f1a885a2de8e4af4b7ec02a52458ebc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:08.966003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:08.981374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 804/1618] 10.222.0.33 () {58 vars in 1059 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:08.982611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552278f-cacf-4255-97e5-720498287d12 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:09.067227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:09.076963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 815/1619] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:09.078177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:09.097540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92474de5-713a-4670-9766-9c6b06e3e5d3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:09.108988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:09.109863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.110235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.110687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.121146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e552278f-cacf-4255-97e5-720498287d12 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:09.126175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:09.126826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.127184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.127498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.165618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:09.216889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:09.219010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:09.220176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:09.223395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:09.232467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 816/1620] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:09.233771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:09.236780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 805/1621] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:09.250112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33369cf-5aea-469b-866a-7a701f2075be service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:09.257695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:09.258582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.258969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.259308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.335054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:09.336972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc603cfa-ba42-4f0a-8aa0-041cf25f0403 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:09.361113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 817/1622] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:09.724936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:09.762254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96484e3-192e-43b8-9286-1f57efde5f00 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:09.782557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:09.784055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.784579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.786034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.810272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:09.812262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0a0bb-0711-4590-ac11-0d7c9937a517 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:09.824664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a23-def8-4982-9f73-ee5027ff8af6 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.826094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a23-def8-4982-9f73-ee5027ff8af6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.826552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca54a23-def8-4982-9f73-ee5027ff8af6 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.828511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 818/1623] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 13:35:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:09.847548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d75c6-2ecd-459a-960e-936bdeaa1a29 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.848093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d75c6-2ecd-459a-960e-936bdeaa1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.848634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833d75c6-2ecd-459a-960e-936bdeaa1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.863723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 806/1624] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:09.895117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-833d75c6-2ecd-459a-960e-936bdeaa1a29 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:09.898674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-833d75c6-2ecd-459a-960e-936bdeaa1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:09.942077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca117ba-5623-40e3-a46a-9aa4c6c39f34 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.942645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca117ba-5623-40e3-a46a-9aa4c6c39f34 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.943140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca117ba-5623-40e3-a46a-9aa4c6c39f34 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.944759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 807/1625] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 13:35:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:09.950991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca117ba-5623-40e3-a46a-9aa4c6c39f34 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:09.953957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 819/1626] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:09.955891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:09.956180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:09.956456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:09.984377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:09.985667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:10.013624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 820/1627] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:35:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:10.034079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:10.129080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca117ba-5623-40e3-a46a-9aa4c6c39f34 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:10.142823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:10.146896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:10.146896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:10.148894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:10.172149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:10.174245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:10.198965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39586d7-88a6-49c4-8ea0-c471728aad68 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:10.203605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:10.204289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:10.204761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:10.205041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:10.216918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:10.219417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc0092-cc96-4fd7-b075-cd0b03df5f6e service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:10.231931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 808/1628] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:10.243134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 821/1629] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:35:11.680638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:11.694163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adcdcb-f0f2-4c8d-b1aa-ea3be2006cee service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:11.705558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:11.708349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:11.710544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:11.710710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] PATH_INFO: `/v3/users/a6ef5f1cd8e64a86bc691aee50fc699d` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:11.716247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ef5f1cd8e64a86bc691aee50fc699d)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:11.718800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:11.837256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ef5f1cd8e64a86bc691aee50fc699d'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:11.849526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ef5f1cd8e64a86bc691aee50fc699d'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:11.862248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ef5f1cd8e64a86bc691aee50fc699d'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:11.864138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ef5f1cd8e64a86bc691aee50fc699d'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:11.865169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ef5f1cd8e64a86bc691aee50fc699d'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:11.874955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:11.901789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 809/1630] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:11 2022] DELETE /identity/v3/users/a6ef5f1cd8e64a86bc691aee50fc699d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:11.904023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:11.904538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:11.904904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:11.939941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:11.941892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:12.078740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:12.093051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 822/1631] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:12.113832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:12.116125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:12.258439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9455be55-86ec-4d7b-84d9-baa7438bd1e4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:12.268118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:12.271475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:12.271936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:12.272255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:12.298394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:12.300194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65219913-546c-4888-bed5-f53c2e87bfbe service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:12.319129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 810/1632] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:14.380643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:14.387199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:14.423947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be251c-a4d1-4e1c-879d-7e19f89dbc85 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:14.437743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:14.438852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:14.438852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:14.439303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] PATH_INFO: `/v3/projects/0367800b12d7428b9a58cff37f5fc501` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:14.440944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0367800b12d7428b9a58cff37f5fc501)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:14.443508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:14.545423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0367800b12d7428b9a58cff37f5fc501', 'request_id': 'req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:14.569626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0367800b12d7428b9a58cff37f5fc501', 'request_id': 'req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:14.598166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0367800b12d7428b9a58cff37f5fc501', 'request_id': 'req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:14.626440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] The token cache is being invalidate because project 0367800b12d7428b9a58cff37f5fc501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:14.627820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:14.628805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5fb9b1ff-7f3a-4665-aec3-fd6d19a631fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:14.630787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 823/1633] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:14 2022] DELETE /identity/v3/projects/0367800b12d7428b9a58cff37f5fc501 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:14.635946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:14.637296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:14.637718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:14.705068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:14.706416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:14.882319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 811/1634] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:14.890905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:14.891676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:14.892194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:14.931750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:14.951371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:14.967134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 824/1635] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:14.970579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:14.985582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73153e59-ddd2-4418-9e47-720e3e7248d2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:14.996849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:14.997634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:14.998194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:14.998572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:14.999402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:14.999846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.001936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.035209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 812/1636] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:15.037485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.062792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7eb079-60cb-4376-a0be-6e805f1b2af3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.069200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.080712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.081407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.081945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.082785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.085495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.152978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 825/1637] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:15 2022] POST /identity/v3/projects => generated 403 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:35:15.157862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.187034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672edd4-6802-4211-8113-6749fd745342 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.227640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.229911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:15.314504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.316596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.317862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.317862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.318526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.320898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.360430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 813/1638] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:15 2022] POST /identity/v3/users => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:15.362203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.373095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:15.378610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508f8c4-cefb-4ebe-9f9c-43cf2932bc5e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.386962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.395990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a816536-fda5-4f3c-ae98-0ced5d22345a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.403329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.404571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.405316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.409049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.411984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.413069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.413360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.413569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.413677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] PATH_INFO: `/v3/users/fa4870e52d734e64a2e07f82a4b59c71` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.420450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa4870e52d734e64a2e07f82a4b59c71)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.421547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: WARNING py.warnings [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:15.421547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: warnings.warn(msg) Aug 08 13:35:15.421547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: Aug 08 13:35:15.422837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.457630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 814/1639] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:15.459149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.495196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533022cc-5dea-4839-a445-c8ad96fe0a35 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.504382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.519497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.520052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.521351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.525263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.531467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.559421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 815/1640] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:15.563411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:15.585049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717a081-ef41-4ac8-9428-51d131ad526b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:15.593145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:15.594181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.596214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.596772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] PATH_INFO: `/v3/projects/ce2914ac6d914caebf7b94b388c6b8fe/users/3bd283b752794c49a7f33416032e949e/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.629622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2914ac6d914caebf7b94b388c6b8fe, user_id=3bd283b752794c49a7f33416032e949e, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:15.633748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe382ad-c390-4aec-bf71-eb76ee9fadb8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:15.635952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa4870e52d734e64a2e07f82a4b59c71'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:15.676638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 816/1641] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:15 2022] PUT /identity/v3/projects/ce2914ac6d914caebf7b94b388c6b8fe/users/3bd283b752794c49a7f33416032e949e/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:15.680955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa4870e52d734e64a2e07f82a4b59c71'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:15.681735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa4870e52d734e64a2e07f82a4b59c71'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:15.682405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa4870e52d734e64a2e07f82a4b59c71'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:15.684722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:15.688615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.688949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.690264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.702623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-2bd97dfb-f4dd-4681-b46f-b5ddd2829031 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa4870e52d734e64a2e07f82a4b59c71'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:15.751385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:15.751385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:15.774236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 826/1642] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:15 2022] DELETE /identity/v3/users/fa4870e52d734e64a2e07f82a4b59c71 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:15.780792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:15.783011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:15.783331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:15.842587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:15.852041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:15.950293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 817/1643] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:16.054800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 827/1644] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:16.112225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:16.114680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:16.118017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:16.125006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:16.362679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72deac79-7932-4614-bc60-1482cdbd3668 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:16.366258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ebb95-e330-4d34-8f3a-a55b189a81ad None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:16.372070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:16.372725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:16.373006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:16.373277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:16.377056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:16.377847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:16.378136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:16.378379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:16.397621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:16.399186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12394ec2-c173-4fc0-ad75-aecad9f189c6 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:16.412509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:16.420629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a293a31-67d6-41ff-981e-db2476df002c service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:16.444997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 818/1645] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:16.447498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 828/1646] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:17.592842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:17.594034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:17.594224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:17.664566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] MFA Rules not processed for user `c93191a5e3c7476f926b56424ccec4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:17.669346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:17.768271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:17.815636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 819/1647] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:17.816402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:17.817198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:17.817618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:17.845455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] MFA Rules not processed for user `c93191a5e3c7476f926b56424ccec4ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:17.847207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:17.856851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 829/1648] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:17.871006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:17.872654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:18.058317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed4034-70df-4ef7-9035-1fb07eeec1ef None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:18.082596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:18.083431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:18.084368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:18.088597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:18.119446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:18.124238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247b0a1d-475a-43db-8346-020bb39179d8 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:18.158423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 820/1649] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:18.897919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:18.900120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:18.929728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe57845-6b3e-44d1-8dec-0218aacd4e35 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:18.947737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:18.948818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:18.949260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:18.949680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] PATH_INFO: `/v3/projects/a3935a3e3fce40d7a936c74f38002fec` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:18.954698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3935a3e3fce40d7a936c74f38002fec)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:18.958987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:19.097198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3935a3e3fce40d7a936c74f38002fec', 'request_id': 'req-80b2da59-d1d1-424f-b109-5793b24cc1a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:19.114173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3935a3e3fce40d7a936c74f38002fec', 'request_id': 'req-80b2da59-d1d1-424f-b109-5793b24cc1a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:19.136363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3935a3e3fce40d7a936c74f38002fec', 'request_id': 'req-80b2da59-d1d1-424f-b109-5793b24cc1a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:19.139209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:19.197326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] The token cache is being invalidate because project a3935a3e3fce40d7a936c74f38002fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:19.197693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:19.200006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-80b2da59-d1d1-424f-b109-5793b24cc1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:19.207157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 830/1650] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:18 2022] DELETE /identity/v3/projects/a3935a3e3fce40d7a936c74f38002fec => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:19.209161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:19.209624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:19.212255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:19.270208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:19.278974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:19.451662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 821/1651] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:19.461202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:19.461415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:19.461730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:19.508674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:19.510059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:19.560772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:19.563214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 831/1652] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:19.579471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2d093-b6a9-4517-938c-5685e1bf6d14 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:19.600214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:19.600887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:19.601164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:19.601435 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:19.602284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:19.602594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:19.604368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:19.635199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 822/1653] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:19.639614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:19.650070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b1fe1-354c-4752-a36a-b3aec3183424 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:19.664501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:19.666301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:19.668534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:19.668976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:19.671037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:19.676960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:19.750329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 832/1654] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:19 2022] POST /identity/v3/projects => generated 403 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:19.752585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:19.768891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786def-03ff-4146-a8f6-5a6bab0ebebd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:19.921475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:19.923700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:19.924872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:19.925698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:19.927972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:19.933412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:19.966779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:19.986824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 823/1655] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:19 2022] POST /identity/v3/users => generated 302 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:35:19.992028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.022712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da34d57a-4434-436b-a5a7-9394a761266c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:20.050976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:20.052618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.053343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.054057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.055822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:20.073429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:20.109251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 833/1656] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:20.113310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.134477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f15fb3-5952-43e0-b5c2-7aa11e36b3b0 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:20.155347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:20.159706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.160711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.160988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.164853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:20.171350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44efb33-01e3-44cb-9326-e79798461d01 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:20.221588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 824/1657] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:20.225585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.245185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc86f55-d5eb-4232-8f98-77ed153a9f1a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:20.262961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:20.263696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.263978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.264319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] PATH_INFO: `/v3/projects/01113084f00440fcbb2f3232b7fcaf50/users/dfc5d3ea0d394f0c8b18c8822ce85ffa/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.308137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01113084f00440fcbb2f3232b7fcaf50, user_id=dfc5d3ea0d394f0c8b18c8822ce85ffa, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:20.314025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb4653-8682-413e-b1d6-a2a1248b8448 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:20.329800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b6c4-f67f-4fb3-926c-902a34de80c0 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.330485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b6c4-f67f-4fb3-926c-902a34de80c0 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.330977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f37b6c4-f67f-4fb3-926c-902a34de80c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.361062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 834/1658] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:20 2022] PUT /identity/v3/projects/01113084f00440fcbb2f3232b7fcaf50/users/dfc5d3ea0d394f0c8b18c8822ce85ffa/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:20.369357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9147-d71a-409b-97bc-13e61933961b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.369828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9147-d71a-409b-97bc-13e61933961b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.370208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80af9147-d71a-409b-97bc-13e61933961b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.394222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5f37b6c4-f67f-4fb3-926c-902a34de80c0 None None] MFA Rules not processed for user `3bd283b752794c49a7f33416032e949e`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:20.396108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5f37b6c4-f67f-4fb3-926c-902a34de80c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:20.426788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-80af9147-d71a-409b-97bc-13e61933961b None None] MFA Rules not processed for user `dfc5d3ea0d394f0c8b18c8822ce85ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:20.430984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-80af9147-d71a-409b-97bc-13e61933961b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:20.577957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 825/1659] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:20.580602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.581816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.583409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.588300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 826/1660] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 13:35:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:20.589418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.597471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:20.615402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 835/1661] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:20.618382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.620900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.621732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.728074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:20.731448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:20.776825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f7074-6e3b-4337-aa15-d3ab7f739395 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:20.791608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:20.792430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:20.792683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:20.793081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:20.795198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:20.797296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:20.862553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 827/1662] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/projects => generated 403 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:20.864055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.865385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 836/1663] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:20.867341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:20.897466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fcbe2-8e00-4f2c-b87d-7f6ea43e484a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:20.937454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:21.014636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:21.019976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3f5291-13e3-4a8a-a6c8-4a7ea045541d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.030069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.031167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.031590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.032030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.032782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.036326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.058543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.059535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.059974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.060261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.061436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.067313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.088871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 828/1664] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/projects => generated 403 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:21.089746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.094143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:21.124951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 837/1665] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:20 2022] POST /identity/v3/users => generated 308 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:21.129184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.157677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0098241-591a-40d9-a052-e9347c2619c0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.258397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eba699-ae9e-4f37-b365-02a4be0f8157 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.263770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.264663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.265069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.265496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.275764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.278166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.281631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.282294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.282562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.282834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.283676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.286919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.290189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 829/1666] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:21.291381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.317399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691f3de3-4771-4c38-8418-8c44dfa35a36 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.328709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.331860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.332391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.332829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.334059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.337253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.339721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 838/1667] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:21 2022] POST /identity/v3/users => generated 308 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:21.344087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.353423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 830/1668] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:21.357743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.369561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60280345-7099-4224-895b-10350c6937e5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.373305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c538c65d-fccb-461f-b9df-3e2505724935 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.390171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.390999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.391306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.391398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.392032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.393350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.393750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.394515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.395021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.395781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] PATH_INFO: `/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42/users/73514440055c4024b3c84736ca27538f/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.416212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c6c23d2ef64df29040fe44ff0b6a42, user_id=73514440055c4024b3c84736ca27538f, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.419540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.430618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 839/1669] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:21.430618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.444292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 831/1670] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:21 2022] PUT /identity/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42/users/73514440055c4024b3c84736ca27538f/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:21.447807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.461321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2e63e6-810e-41db-9369-18068b02ecbf admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.575051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f786d97-dcc2-401d-abcb-df0e99c94a05 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.586347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.587246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.588546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.588723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.589661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.589909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.590080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] PATH_INFO: `/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.590206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.590846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.596747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.632384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 832/1671] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:21.649063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1637c4e74a6d401880d9f3b5eee67b4c, user_id=5fc7b8f68d2b43c19b7ba34e9a0cd33a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.649433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.657802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.698395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 840/1672] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:21 2022] PUT /identity/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:21.708591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0528ffd-2cf8-4bd6-b6ab-f30eaf820dfa admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.709537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.735288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:21.896064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e913a-8d59-44c9-9149-d31d3048b7c5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:21.902068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.902750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.903020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.908562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:21.909605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:21.911440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:21.915960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:21.958463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 841/1673] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:21.962594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:21.982636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:21.983368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:21.983650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:21.983930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] PATH_INFO: `/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42/users/73514440055c4024b3c84736ca27538f/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:22.002589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91c6c23d2ef64df29040fe44ff0b6a42, user_id=73514440055c4024b3c84736ca27538f, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:22.013958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6ce398-4490-4a1c-934b-914e8e57acc6 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:22.015530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e99dbda-ce01-453e-9e64-0a435dd60da1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:22.021051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:22.021756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:22.022094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:22.022370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] PATH_INFO: `/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:22.027094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1637c4e74a6d401880d9f3b5eee67b4c, user_id=5fc7b8f68d2b43c19b7ba34e9a0cd33a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:22.033852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:22.047061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 833/1674] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:21 2022] PUT /identity/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42/users/73514440055c4024b3c84736ca27538f/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:22.048121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 842/1675] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:21 2022] PUT /identity/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:22.060453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267d5b-f64d-43ba-b319-fb9d40c3b4a3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:22.060632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267d5b-f64d-43ba-b319-fb9d40c3b4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:22.060942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3267d5b-f64d-43ba-b319-fb9d40c3b4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:22.125816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e3267d5b-f64d-43ba-b319-fb9d40c3b4a3 None None] MFA Rules not processed for user `5fc7b8f68d2b43c19b7ba34e9a0cd33a`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:22.127052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e3267d5b-f64d-43ba-b319-fb9d40c3b4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:22.307440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 834/1676] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:22 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:22.320419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:22.323370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:22.637521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4432f-7920-43f1-be47-64f80da4a9ce admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:22.645929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:22.647058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:22.647571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:22.648163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:22.676586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:22.682829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:22.723038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 843/1677] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:22 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:22.867634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:22.867973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:22.868294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:22.954080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] MFA Rules not processed for user `fce6a7d7dbc94adeb0b03bae34e335a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:22.955689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:23.110804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:23.172081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 835/1678] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:23.349537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:23.352163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:23.419955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7088b-3749-44b6-b3a5-3fd795773d04 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:23.440494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:23.442693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:23.443703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:23.446396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] PATH_INFO: `/v3/users/dfc5d3ea0d394f0c8b18c8822ce85ffa` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:23.450743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfc5d3ea0d394f0c8b18c8822ce85ffa)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:23.463608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:23.576113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:23.643816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:23.658742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4894784-d12c-4c48-82ad-c1f646977cd2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:23.676719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:23.677481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:23.677808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:23.678135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:23.699444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfc5d3ea0d394f0c8b18c8822ce85ffa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:23.702481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:23.708588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b8a50-b5d1-4fc1-b936-8e1eafa8837b service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:23.736268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 836/1679] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:23.738876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfc5d3ea0d394f0c8b18c8822ce85ffa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:23.742854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfc5d3ea0d394f0c8b18c8822ce85ffa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:23.743872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfc5d3ea0d394f0c8b18c8822ce85ffa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:23.770484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfc5d3ea0d394f0c8b18c8822ce85ffa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:23.869299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 844/1680] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:23 2022] DELETE /identity/v3/users/dfc5d3ea0d394f0c8b18c8822ce85ffa => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:23.874641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2588d-ac0d-4fc1-8c5e-1b2ebce23c9d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:23.875288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2588d-ac0d-4fc1-8c5e-1b2ebce23c9d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:23.875664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2588d-ac0d-4fc1-8c5e-1b2ebce23c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:23.940179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-19d2588d-ac0d-4fc1-8c5e-1b2ebce23c9d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:23.941235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-19d2588d-ac0d-4fc1-8c5e-1b2ebce23c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:24.156185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 837/1681] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:24.167300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:24.176450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:24.420611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd84a31-efc8-4fd3-8b24-8d7df1a75305 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:24.427575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:24.428613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:24.429101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:24.429566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:24.433147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c58c2-a0ae-4708-93ee-a8b030fc84e8 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:24.434494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c58c2-a0ae-4708-93ee-a8b030fc84e8 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:24.435057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c58c2-a0ae-4708-93ee-a8b030fc84e8 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:24.438009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 838/1682] 10.222.0.33 () {54 vars in 961 bytes} [Mon Aug 8 13:35:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:24.448303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:24.448725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:24.449087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:24.484865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:24.487804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:24.500604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:24.503557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:24.560557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 845/1683] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:24.636578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:24.645123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 839/1684] 10.222.0.33 () {58 vars in 1058 bytes} [Mon Aug 8 13:35:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:24.647159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:24.670337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:24.688099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f027faad-48f3-4c0b-95a0-eb4af12d1c66 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:24.699524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:24.703546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:24.703546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:24.703546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] PATH_INFO: `/v3/users/f1a885a2de8e4af4b7ec02a52458ebc0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:24.705134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1a885a2de8e4af4b7ec02a52458ebc0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:24.715129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7a29b8-ede7-4579-bc6c-8b7171a2b14e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:24.716219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:24.735753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:24.737409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:24.745162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:24.745806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:24.752798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:24.808887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:24.812157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06087f03-7f5c-4cd5-a7f4-329a71a02614 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:24.844353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1a885a2de8e4af4b7ec02a52458ebc0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:24.856601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1a885a2de8e4af4b7ec02a52458ebc0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:24.858017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1a885a2de8e4af4b7ec02a52458ebc0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:24.858729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1a885a2de8e4af4b7ec02a52458ebc0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:24.893725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1a885a2de8e4af4b7ec02a52458ebc0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:24.909164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 840/1685] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:24.999413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 846/1686] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:24 2022] DELETE /identity/v3/users/f1a885a2de8e4af4b7ec02a52458ebc0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:25.006874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:25.007204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:25.008357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:25.047468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:25.055003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:25.280766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 841/1687] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:25.361980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:25.364607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:25.517978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:25.598056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b6f97-ea78-4d99-9141-d633126d7d80 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:25.626632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:25.628267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:25.629348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:25.629969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:25.701679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:25.704597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:25.741485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 847/1688] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:26.472051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:26.473673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:26.490675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:26.497574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5412f0-2278-47f6-8740-a26c00b65a8f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:26.536873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:26.545476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:26.545973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:26.546456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] PATH_INFO: `/v3/projects/01113084f00440fcbb2f3232b7fcaf50` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:26.567831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01113084f00440fcbb2f3232b7fcaf50)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:26.578478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: WARNING py.warnings [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 08 13:35:26.578478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: warnings.warn(msg) Aug 08 13:35:26.578478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: Aug 08 13:35:26.584688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:26.805089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01113084f00440fcbb2f3232b7fcaf50', 'request_id': 'req-217306d6-46fe-4d26-892a-628f15f45ef7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:26.825677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01113084f00440fcbb2f3232b7fcaf50', 'request_id': 'req-217306d6-46fe-4d26-892a-628f15f45ef7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:26.862043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01113084f00440fcbb2f3232b7fcaf50', 'request_id': 'req-217306d6-46fe-4d26-892a-628f15f45ef7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:26.926034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] The token cache is being invalidate because project 01113084f00440fcbb2f3232b7fcaf50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:26.926715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:26.929512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-217306d6-46fe-4d26-892a-628f15f45ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:26.940145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 842/1689] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:26 2022] DELETE /identity/v3/projects/01113084f00440fcbb2f3232b7fcaf50 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:26.941175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:26.983118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e340d33-f485-4c81-a6ba-9c69a0099cc0 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:27.239440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:27.241960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:27.244200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:27.244676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] PATH_INFO: `/v3/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:27.248267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc7b8f68d2b43c19b7ba34e9a0cd33a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:27.266133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:27.280347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1aa46-d48b-4041-b3bc-5cd05afd2dd1 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:27.281151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1aa46-d48b-4041-b3bc-5cd05afd2dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:27.281698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1aa46-d48b-4041-b3bc-5cd05afd2dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:27.443568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-afe1aa46-d48b-4041-b3bc-5cd05afd2dd1 None None] MFA Rules not processed for user `518d6948777944fd8214dc6d778a9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:27.450177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-afe1aa46-d48b-4041-b3bc-5cd05afd2dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:27.530643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc7b8f68d2b43c19b7ba34e9a0cd33a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:27.555917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc7b8f68d2b43c19b7ba34e9a0cd33a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:27.560138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc7b8f68d2b43c19b7ba34e9a0cd33a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:27.580372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc7b8f68d2b43c19b7ba34e9a0cd33a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:27.641519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc7b8f68d2b43c19b7ba34e9a0cd33a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:27.721471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 848/1690] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:26 2022] DELETE /identity/v3/users/5fc7b8f68d2b43c19b7ba34e9a0cd33a => generated 0 bytes in 778 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:27.721471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:27.729755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:27.746219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 843/1691] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:27.768586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:27.769870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:27.769870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:27.816321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] MFA Rules not processed for user `518d6948777944fd8214dc6d778a9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:27.817682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:28.071493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:28.118940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 844/1692] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:28.134836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:28.137156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:28.186104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7eaa-984e-42c4-be9b-e753b400ca12 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:28.226216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:28.230961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:28.231866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:28.232584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] PATH_INFO: `/v3/projects/7443e956a2d7470f878fcb96a99e6e40` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:28.236603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7443e956a2d7470f878fcb96a99e6e40)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:28.255873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:28.274806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:28.440457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7443e956a2d7470f878fcb96a99e6e40', 'request_id': 'req-f96d536a-302a-469c-976b-3928a25248a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.453290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7443e956a2d7470f878fcb96a99e6e40', 'request_id': 'req-f96d536a-302a-469c-976b-3928a25248a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.499895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7443e956a2d7470f878fcb96a99e6e40', 'request_id': 'req-f96d536a-302a-469c-976b-3928a25248a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.538101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] The token cache is being invalidate because project 7443e956a2d7470f878fcb96a99e6e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:28.539101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.540482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f96d536a-302a-469c-976b-3928a25248a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.542809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 849/1693] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:27 2022] DELETE /identity/v3/projects/7443e956a2d7470f878fcb96a99e6e40 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:28.550653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:28.551366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:28.551884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:28.554536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4823f9b5-90d7-4e81-aa82-677e0eff1485 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:28.628290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:28.632534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:28.802783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 850/1694] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:28.804745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:28.809290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:28.833036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344bf66-1e89-4f2a-871a-80a64b410d44 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:28.840696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:28.841535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:28.841817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:28.842167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] PATH_INFO: `/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:28.843648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1637c4e74a6d401880d9f3b5eee67b4c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:28.845731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:28.951972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:28.953098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:28.953571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:28.954044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:28.964417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1637c4e74a6d401880d9f3b5eee67b4c', 'request_id': 'req-e9429600-8539-4fce-be74-2e45f3a66e17'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:28.968652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:28.974066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1637c4e74a6d401880d9f3b5eee67b4c', 'request_id': 'req-e9429600-8539-4fce-be74-2e45f3a66e17'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:29.016128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1637c4e74a6d401880d9f3b5eee67b4c', 'request_id': 'req-e9429600-8539-4fce-be74-2e45f3a66e17'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:29.082770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] The token cache is being invalidate because project 1637c4e74a6d401880d9f3b5eee67b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:29.084357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:29.088949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-e9429600-8539-4fce-be74-2e45f3a66e17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:29.095899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 851/1695] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:28 2022] DELETE /identity/v3/projects/1637c4e74a6d401880d9f3b5eee67b4c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:29.102020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.102561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.104643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.133760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:29.203399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:29.204509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:29.206430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9292ec9-6c2e-4aa3-9f6d-63c7cd09c55e service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:29.209298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:29.229881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 845/1696] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:28 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 1096 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:35:29.231705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0b4cd-118b-4f6d-bf96-985a14c9b134 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.234291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0b4cd-118b-4f6d-bf96-985a14c9b134 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.234745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0b4cd-118b-4f6d-bf96-985a14c9b134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.294182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-70f0b4cd-118b-4f6d-bf96-985a14c9b134 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:29.295927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-70f0b4cd-118b-4f6d-bf96-985a14c9b134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:29.474895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:29.513804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 846/1697] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:29.516053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.516338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.516616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.566964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 852/1698] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:29.572130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:29.612891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:29.618877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:29.645205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 847/1699] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:29.646401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:29.661229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035752c5-1802-40e9-ab46-d5f308cde035 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:29.665442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12509b-ae48-48ef-865b-7c8d8624c040 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:29.678391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:29.681158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.681880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.683728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.684600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:29.686632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:29.690992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:29.691244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:29.691962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.692373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.692885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.699771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:29.702560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:29.713553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:29.730599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 848/1700] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:29.731886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:29.755247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d7281e-9a1f-4473-a8ed-4e8462028be9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:29.769123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:29.769938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.770314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.770682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.771531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:29.773658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:29.774712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 853/1701] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:29.783057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:29.836217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e8b249-0c3c-4abe-a06c-a88bd14bf8fc admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:29.847329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 849/1702] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:29.848677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:29.852036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:29.853163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:29.853321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:29.853452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:29.854121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:29.856163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:29.948182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 854/1703] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/projects => generated 369 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:29.959117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:30.043631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9f182-9933-4218-9877-6b88d87b113e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:30.181585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b3ee0-b946-4c36-a659-dee3be4da242 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:30.196041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:30.196711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.196991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.197344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.198017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:30.200971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:30.275490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 855/1704] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/users => generated 285 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:30.279279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:30.327806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deba94-8aed-4e0e-ad5b-43ae20659fa8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:30.348804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:30.349978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.350380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.350768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.353466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:30.362936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcbe36-45d6-497b-9fe6-6725e8f0bb86 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:30.396848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:30.397854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.400217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.401609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.402510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:30.405254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 856/1705] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:30.409380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.409980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.410453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.414416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7caf7a9-93b3-470a-8196-8432c8728a23 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:30.519398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 850/1706] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:29 2022] POST /identity/v3/users => generated 291 bytes in 671 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:30.521832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8229bca-429e-4530-bbd2-f6e0c8d5c6c0 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.522426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8229bca-429e-4530-bbd2-f6e0c8d5c6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.522936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8229bca-429e-4530-bbd2-f6e0c8d5c6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.549265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] MFA Rules not processed for user `651659e941044454985b2f866b791b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:30.554133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:30.757009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:30.767635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b8229bca-429e-4530-bbd2-f6e0c8d5c6c0 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:30.769041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b8229bca-429e-4530-bbd2-f6e0c8d5c6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:30.833884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 857/1707] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 428 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:30.835275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:30.851805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:30.858810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f37973-2622-454b-8fca-f54df5c34b60 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:30.872545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:30.873639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.874169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.874863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.875813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:30.877870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:30.898452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 858/1708] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:30.899826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:30.929870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69821b3-e708-4ba7-a8e2-d3bf26e687bb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:30.944271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:30.945335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.945857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.946353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.947449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 851/1709] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:30.947710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:30.949408 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:30.950152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:30.950152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:30.951314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:30.977627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 859/1710] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:30.977627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:30.998402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27392d99-89a4-448d-a1a6-e4ba5461aa43 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:31.024575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] MFA Rules not processed for user `73514440055c4024b3c84736ca27538f`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:31.025964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:31.027286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:31.028113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:31.028505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:31.028886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] PATH_INFO: `/v3/projects/78eb3b2dc06e409dab376b3a227779ed/users/e549d616fcf2453b931444b0fffecf7b/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:31.082318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78eb3b2dc06e409dab376b3a227779ed, user_id=e549d616fcf2453b931444b0fffecf7b, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:31.085471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:31.114301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 860/1711] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:30 2022] PUT /identity/v3/projects/78eb3b2dc06e409dab376b3a227779ed/users/e549d616fcf2453b931444b0fffecf7b/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:31.115693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:31.231684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 852/1712] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:30 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:31.234879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:31.240651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:31.314570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394bfd1d-4167-495a-8f3c-6aff6aa75991 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:31.329710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:31.330400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:31.330690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:31.330961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:31.331682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:31.333622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543327d-fac0-4772-874f-8bce10674013 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:31.358391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 861/1713] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:31 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:31.359973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e766-f574-471f-907d-2bd0f88cc004 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:31.360259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e766-f574-471f-907d-2bd0f88cc004 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:31.360530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588e766-f574-471f-907d-2bd0f88cc004 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:31.370156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 862/1714] 10.222.0.33 () {54 vars in 912 bytes} [Mon Aug 8 13:35:31 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:31.375579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:31.375579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:31.375579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:31.451237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] MFA Rules not processed for user `e549d616fcf2453b931444b0fffecf7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:31.452560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:31.575426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a2b49-ba14-4cea-bd7c-19acf6a0e6f7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:31.633368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 863/1715] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:31.634027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:31.641631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:31.704851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:31.807258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:31.975318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:32.000564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.002455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.004496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:32.076478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfa1039-5234-48d0-beb3-ba53749e774d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:32.097233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:32.099325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.099775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.101027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:32.252685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:32.256012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:32.271414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:32.277411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9c743-6b8c-4759-85c5-640ae6b2d5f3 service swift] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:32.296710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 853/1716] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1062 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:32.297357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:32.339271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f5e101-26c8-44e0-b584-298efc30a1b0 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:32.356759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:32.357659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 864/1717] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:35:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 729 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:35:32.358076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.358283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.358775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] PATH_INFO: `/v3/projects/c149092b47be48099974973273a4ff68/users/a5d57cb53fe344719f6b860b661f5ef3/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:32.360532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f554-09c7-4a58-a7db-b8700a909f67 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.361127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f554-09c7-4a58-a7db-b8700a909f67 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.361676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d5f554-09c7-4a58-a7db-b8700a909f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:32.514302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c149092b47be48099974973273a4ff68, user_id=a5d57cb53fe344719f6b860b661f5ef3, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:32.524969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:32.583504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-22d5f554-09c7-4a58-a7db-b8700a909f67 None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:32.585010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-22d5f554-09c7-4a58-a7db-b8700a909f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:32.591717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 854/1718] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:32 2022] PUT /identity/v3/projects/c149092b47be48099974973273a4ff68/users/a5d57cb53fe344719f6b860b661f5ef3/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:32.622669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:32.900486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1db23e-5822-404c-a141-26da6c8d2461 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:32.919359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:32.920160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.920443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.920717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:32.921376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:32.923488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e992cc-4db9-4e28-ba23-b91709a3f282 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:32.936081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 865/1719] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 577 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:32.939796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:32.942236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:32.943642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:33.135543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:33.150459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:33.177358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 855/1720] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:32 2022] POST /identity/v3/projects => generated 367 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:33.180439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 866/1721] 10.222.0.48 () {58 vars in 1015 bytes} [Mon Aug 8 13:35:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:33.185239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:33.190125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:33.190125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:33.190567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:33.277149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a803a-1628-49a5-8201-4403c1ddfb21 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:33.444603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] MFA Rules not processed for user `a5d57cb53fe344719f6b860b661f5ef3`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:33.446520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:33.531119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:33.622943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:33.622943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:33.628119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:33.628805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:33.629871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:33.633656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:33.709896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:33.744188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 867/1722] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:33 2022] POST /identity/v3/users => generated 290 bytes in 561 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:33.753661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:33.760430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:33.868455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 856/1723] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 690 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:33.878038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:33.886055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:34.036147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b42ef-4486-40ac-810a-9c1f7bb0b8a3 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.049699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.051033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.051501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.051937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.074371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.076424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.097443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 868/1724] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:34.101729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.134693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a29267b-0b2b-4c36-9adc-883b67cc224f service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.150721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.151693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.152146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.152575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.153646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.157887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.161727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dafb44d-6943-445b-8d49-bb53a2ee4b91 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.173561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.174296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.174637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.174915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.191306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 869/1725] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:34.192335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.196710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.198597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.213826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a938-7718-41bc-bcd6-b3e75e2505dd admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.223549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 857/1726] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:34.225461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.228080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.231161 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.232207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.233098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.234643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.240814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.252511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ae9bc-d1ac-4fc3-8fd2-19f01cca8ffc service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.278127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.279929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.280166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.280443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] PATH_INFO: `/v3/projects/6872c6e91c7c4c31999544f7bd82e864/users/f2bae2c4ca4d40f78032f07d35b91948/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.308948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 870/1727] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:34 2022] POST /identity/v3/projects => generated 381 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:34.314569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.317742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6872c6e91c7c4c31999544f7bd82e864, user_id=f2bae2c4ca4d40f78032f07d35b91948, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.325350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.360644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 858/1728] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:34 2022] PUT /identity/v3/projects/6872c6e91c7c4c31999544f7bd82e864/users/f2bae2c4ca4d40f78032f07d35b91948/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:34.362089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.589558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b813-b69d-4b34-a645-8da987ee8806 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.670736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b26c9-ce29-42c2-a652-6efb5cf697c0 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.695126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.698530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.701196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.701847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.704401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.711352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.745451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 859/1729] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 384 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:34.747276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:34.747837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.773420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3de269-4f99-4f3e-9347-281fe5164211 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:34.776772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.777850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.778406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.778994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.780120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.786745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:34.787305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.788774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.789901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.790920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] PATH_INFO: `/v3/projects/6872c6e91c7c4c31999544f7bd82e864/users/f2bae2c4ca4d40f78032f07d35b91948/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.795952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6872c6e91c7c4c31999544f7bd82e864, user_id=f2bae2c4ca4d40f78032f07d35b91948, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:34.802678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83a3400-4ef3-4e00-883c-3f0735b411f7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:34.852610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 871/1730] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:34 2022] POST /identity/v3/users => generated 297 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:34.856038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:34.870508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 860/1731] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:34 2022] PUT /identity/v3/projects/6872c6e91c7c4c31999544f7bd82e864/users/f2bae2c4ca4d40f78032f07d35b91948/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:34.881187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:34.881988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:34.882492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:34.890789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dfe150-5f16-4632-9546-4876f177b3c2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:35.005388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] MFA Rules not processed for user `f2bae2c4ca4d40f78032f07d35b91948`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:35.007071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:35.139321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:35.139768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.140049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.140321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.140997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:35.142738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:35.169396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 872/1732] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:34 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:35.174126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:35.180350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:35.200064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80beebff-7299-44fd-a227-d159818761ec admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:35.216241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:35.217644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.217929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.218205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] PATH_INFO: `/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3/users/969c93b42aaf4b2396c8a2bd5ea7bf23/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.276124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39f7a3f50b3487c82db1775dc7e68a3, user_id=969c93b42aaf4b2396c8a2bd5ea7bf23, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:35.281022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:35.333167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 873/1733] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:35 2022] PUT /identity/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3/users/969c93b42aaf4b2396c8a2bd5ea7bf23/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:35.335800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 861/1734] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:35.343268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:35.365644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:35.368612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:35.523134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a86ecac-b79f-4bfd-8e94-011897bdc525 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:35.584483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dffa73-7cd6-4e1f-9a14-502b7f662ee4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:35.623963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:35.625917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.626586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.627230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.628376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:35.630774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:35.672286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 874/1735] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:35.674195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:35.708291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a021b58-786d-42e7-a12e-955d2b1ea697 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:35.729616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:35.730332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.730617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.730892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] PATH_INFO: `/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3/users/969c93b42aaf4b2396c8a2bd5ea7bf23/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.738262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f39f7a3f50b3487c82db1775dc7e68a3, user_id=969c93b42aaf4b2396c8a2bd5ea7bf23, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:35.745041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1961572-029f-47b7-a771-8f42069aea5f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:35.792357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 875/1736] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:35 2022] PUT /identity/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3/users/969c93b42aaf4b2396c8a2bd5ea7bf23/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:35.792542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:35.794356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.794521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:35.794693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.794966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.829296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:35.834000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:35.872723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] MFA Rules not processed for user `969c93b42aaf4b2396c8a2bd5ea7bf23`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:35.874084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:35.937026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:36.036562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:36.039012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:36.066823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 862/1737] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 702 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:36.217669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 876/1738] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 424 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:36.248179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:36.250332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:36.283332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c961e41-a890-4265-b58f-4351cd916330 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:36.347316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:36.506493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:36.507349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:36.507913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:36.509135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:36.547691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:36.564447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cbd326-370b-43c6-bab9-ec5e5811d8ad service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:36.596244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 863/1739] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:38.314081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:38.640748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41684a4-d6de-48ad-9b83-8a2d28aa5a3a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:38.661951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402621a1-2479-4e8d-b7c5-9aecb9c10a55 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:38.662912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402621a1-2479-4e8d-b7c5-9aecb9c10a55 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:38.663432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402621a1-2479-4e8d-b7c5-9aecb9c10a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:38.675138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:38.679677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:38.679677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:38.679677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] PATH_INFO: `/v3/users/a5d57cb53fe344719f6b860b661f5ef3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:38.685289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5d57cb53fe344719f6b860b661f5ef3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:38.698609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:38.753571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-402621a1-2479-4e8d-b7c5-9aecb9c10a55 None None] MFA Rules not processed for user `518d6948777944fd8214dc6d778a9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:38.754893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-402621a1-2479-4e8d-b7c5-9aecb9c10a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:38.980271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5d57cb53fe344719f6b860b661f5ef3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:39.064036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5d57cb53fe344719f6b860b661f5ef3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:39.069575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5d57cb53fe344719f6b860b661f5ef3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:39.073973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5d57cb53fe344719f6b860b661f5ef3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:39.091657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7e7b6d9-2ad1-4d5c-a4f7-612563b1ac85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5d57cb53fe344719f6b860b661f5ef3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:39.121687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 864/1740] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:39.121687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326c41-c285-4501-9a13-608a3d3da0d0 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:39.122334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326c41-c285-4501-9a13-608a3d3da0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:39.124293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef326c41-c285-4501-9a13-608a3d3da0d0 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:39.129550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 865/1741] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:35:39 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:39.203527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c96-7652-408a-aa8e-bab789ccd1f5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:39.203750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 877/1742] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:38 2022] DELETE /identity/v3/users/a5d57cb53fe344719f6b860b661f5ef3 => generated 0 bytes in 882 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:39.204143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c96-7652-408a-aa8e-bab789ccd1f5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:39.204595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e5c96-7652-408a-aa8e-bab789ccd1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:39.338062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f00e5c96-7652-408a-aa8e-bab789ccd1f5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:39.340350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f00e5c96-7652-408a-aa8e-bab789ccd1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:39.417843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb13c-2e6c-418b-9b1c-e644c8f8b998 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:39.423004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb13c-2e6c-418b-9b1c-e644c8f8b998 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:39.423666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268cb13c-2e6c-418b-9b1c-e644c8f8b998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:39.502567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-268cb13c-2e6c-418b-9b1c-e644c8f8b998 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:39.503898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-268cb13c-2e6c-418b-9b1c-e644c8f8b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:39.636754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 866/1743] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:39.638596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:39.639066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:39.641780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:39.644553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 867/1744] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:35:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:39.687955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:39.767734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:39.770787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:39.892563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 878/1745] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:35:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:39.932763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:39.933025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:39.933308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:39.980280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47989f5a-8ba6-4614-a668-56fd82f160cf None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:40.018880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:40.020487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:40.021906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:40.023248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:40.025062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:40.026953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:40.100555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:40.102011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51e403e-8fa3-431c-8e52-e2ffc1edf8b9 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:40.157164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 868/1746] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:35:40.223634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:40.227061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:40.227784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:40.331845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] MFA Rules not processed for user `fce6a7d7dbc94adeb0b03bae34e335a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:40.342436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:40.483060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:40.485910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 879/1747] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:35:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 554 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:40.488041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:40.505462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:40.590727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d93613-7e63-4746-a373-45f0e691a61f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:40.650186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:40.677277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:40.685840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:40.687012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:40.750016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:40.754393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:40.812922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 880/1748] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:40.819031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:40.819617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 869/1749] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 597 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:40.824290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:40.891663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccda922-4583-4b55-a65b-f46793c6fc16 service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:40.917730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:40.918426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:40.919959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:40.919959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:40.963238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:40.970089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bc0fc2-8e2f-4b0a-95fd-15a343ece015 service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:41.100083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 881/1750] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:41.866113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:41.872350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:41.915540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad80314-a075-43b9-b8de-78f4d6d0986d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:41.943398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:41.944576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:41.945197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:41.945730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] PATH_INFO: `/v3/users/e549d616fcf2453b931444b0fffecf7b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:41.955425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e549d616fcf2453b931444b0fffecf7b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:41.962153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:42.271152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e549d616fcf2453b931444b0fffecf7b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:42.311588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e549d616fcf2453b931444b0fffecf7b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:42.335060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e549d616fcf2453b931444b0fffecf7b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:42.335780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e549d616fcf2453b931444b0fffecf7b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:42.336802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e549d616fcf2453b931444b0fffecf7b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:42.418997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 870/1751] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:41 2022] DELETE /identity/v3/users/e549d616fcf2453b931444b0fffecf7b => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:42.423847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4db7b1-1aa4-4968-8b82-b8905606dd4e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:42.424755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4db7b1-1aa4-4968-8b82-b8905606dd4e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:42.424755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4db7b1-1aa4-4968-8b82-b8905606dd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:42.497543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-db4db7b1-1aa4-4968-8b82-b8905606dd4e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:42.502692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-db4db7b1-1aa4-4968-8b82-b8905606dd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:42.727776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 882/1752] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:42.788839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:42.790116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:43.089536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e4297-3e3d-41c2-84be-f98f7e877e64 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:43.099178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:43.101603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:43.102467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:43.103302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:43.135771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:43.144751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b174df-1df6-41f5-9a01-4d046c0d9b0b service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:43.166184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 871/1753] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:44.092419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:44.093450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:44.093955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:44.172764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] MFA Rules not processed for user `651659e941044454985b2f866b791b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:44.178424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:44.328963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:44.408706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 883/1754] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:44.610352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:44.611033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:44.611276 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:44.712541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] MFA Rules not processed for user `c29ce05ebc6e4672bb9557f751a48557`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:44.717617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:44.896810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:45.013184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 872/1755] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 405 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:45.927243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:45.933914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:45.996899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e2404-bfc9-4ddb-9b9f-3be11b275f8c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:46.030036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:46.032971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:46.036416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:46.037674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] PATH_INFO: `/v3/projects/c149092b47be48099974973273a4ff68` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:46.042708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c149092b47be48099974973273a4ff68)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:46.054849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:46.200607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c149092b47be48099974973273a4ff68', 'request_id': 'req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.215772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c149092b47be48099974973273a4ff68', 'request_id': 'req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.238607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c149092b47be48099974973273a4ff68', 'request_id': 'req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.254234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] The token cache is being invalidate because project c149092b47be48099974973273a4ff68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:46.255067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.256099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.258292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 884/1756] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:45 2022] DELETE /identity/v3/projects/c149092b47be48099974973273a4ff68 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:46.262903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:46.277143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb8493-c43a-4f44-a204-b6aaf1c8d5a7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:46.592812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:46.594131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:46.594741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:46.595528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] PATH_INFO: `/v3/users/969c93b42aaf4b2396c8a2bd5ea7bf23` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:46.597668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=969c93b42aaf4b2396c8a2bd5ea7bf23)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:46.614795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:46.769474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:46.771236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:46.824338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25ce2b-3092-43b4-b0d7-fde03be4ecf8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:46.860683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '969c93b42aaf4b2396c8a2bd5ea7bf23'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.868711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:46.869659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:46.872803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:46.873146 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] PATH_INFO: `/v3/users/3bd283b752794c49a7f33416032e949e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:46.881168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bd283b752794c49a7f33416032e949e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:46.888116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:46.893569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '969c93b42aaf4b2396c8a2bd5ea7bf23'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.933983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '969c93b42aaf4b2396c8a2bd5ea7bf23'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.934748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '969c93b42aaf4b2396c8a2bd5ea7bf23'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:46.935552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '969c93b42aaf4b2396c8a2bd5ea7bf23'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.038663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 873/1757] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:46 2022] DELETE /identity/v3/users/969c93b42aaf4b2396c8a2bd5ea7bf23 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:47.097417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bd283b752794c49a7f33416032e949e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.127099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bd283b752794c49a7f33416032e949e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.129980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bd283b752794c49a7f33416032e949e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.131509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bd283b752794c49a7f33416032e949e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.156189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bd283b752794c49a7f33416032e949e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:47.197157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 885/1758] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:46 2022] DELETE /identity/v3/users/3bd283b752794c49a7f33416032e949e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:47.253118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:47.254583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:47.530681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:47.585822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4718a32-c4e6-46ef-a992-26a9684728e9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:47.747608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:47.750265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:47.750698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:47.751210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] PATH_INFO: `/v3/projects/78eb3b2dc06e409dab376b3a227779ed` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:47.753378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78eb3b2dc06e409dab376b3a227779ed)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:47.755945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:47.815701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:47.818993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:47.989556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78eb3b2dc06e409dab376b3a227779ed', 'request_id': 'req-a4b4686e-7261-44d5-9454-1ce77719ede0'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.005209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78eb3b2dc06e409dab376b3a227779ed', 'request_id': 'req-a4b4686e-7261-44d5-9454-1ce77719ede0'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.061845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78eb3b2dc06e409dab376b3a227779ed', 'request_id': 'req-a4b4686e-7261-44d5-9454-1ce77719ede0'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.066489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:48.107161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] The token cache is being invalidate because project 78eb3b2dc06e409dab376b3a227779ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:48.108415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.110609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.121747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 874/1759] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:47 2022] DELETE /identity/v3/projects/78eb3b2dc06e409dab376b3a227779ed => generated 0 bytes in 863 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:48.124233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:48.127799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:48.241639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9f80b2-ae3b-43cc-b972-fce5df415339 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:48.493182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b4686e-7261-44d5-9454-1ce77719ede0 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:48.512531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:48.515631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:48.518746 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:48.519301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] PATH_INFO: `/v3/users/f2bae2c4ca4d40f78032f07d35b91948` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:48.524227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2bae2c4ca4d40f78032f07d35b91948)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:48.532188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:48.575320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:48.576939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:48.577598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:48.579299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:48.582779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:48.714391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:48.767724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2bae2c4ca4d40f78032f07d35b91948'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.787834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2bae2c4ca4d40f78032f07d35b91948'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.807324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2bae2c4ca4d40f78032f07d35b91948'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.807947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2bae2c4ca4d40f78032f07d35b91948'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.809031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2bae2c4ca4d40f78032f07d35b91948'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:48.866556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 875/1760] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:48 2022] DELETE /identity/v3/users/f2bae2c4ca4d40f78032f07d35b91948 => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:48.876141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:48.876141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:48.919341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 886/1761] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 1095 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:49.801099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:49.802654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:49.957116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceedec2-753f-4e78-ac5c-f1abba122058 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:49.967395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:49.968101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:49.968398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:49.968664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] PATH_INFO: `/v3/projects/ce2914ac6d914caebf7b94b388c6b8fe` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:49.970667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2914ac6d914caebf7b94b388c6b8fe)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:49.978325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:50.095843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2914ac6d914caebf7b94b388c6b8fe', 'request_id': 'req-c023d145-3a01-4302-8555-c0e76bd82351'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.146378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2914ac6d914caebf7b94b388c6b8fe', 'request_id': 'req-c023d145-3a01-4302-8555-c0e76bd82351'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.165459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2914ac6d914caebf7b94b388c6b8fe', 'request_id': 'req-c023d145-3a01-4302-8555-c0e76bd82351'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.187915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] The token cache is being invalidate because project ce2914ac6d914caebf7b94b388c6b8fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:50.188425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.189812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.194495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 876/1762] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:49 2022] DELETE /identity/v3/projects/ce2914ac6d914caebf7b94b388c6b8fe => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:50.200166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:50.202612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:50.216870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:50.221699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:50.243019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a226b1-1c21-48e4-b311-c11d4ac91f3a service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:50.331648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:50.453675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023d145-3a01-4302-8555-c0e76bd82351 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:50.465609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:50.466539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:50.467001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:50.467454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] PATH_INFO: `/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:50.470490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f39f7a3f50b3487c82db1775dc7e68a3)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:50.472802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:50.551230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:50.552015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:50.552339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:50.552614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] PATH_INFO: `/v3/users/73514440055c4024b3c84736ca27538f` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:50.555762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=73514440055c4024b3c84736ca27538f)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:50.557894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:50.596259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:50.615383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f39f7a3f50b3487c82db1775dc7e68a3', 'request_id': 'req-c896e6be-e976-4d9b-8330-8ce204f17481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.637706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f39f7a3f50b3487c82db1775dc7e68a3', 'request_id': 'req-c896e6be-e976-4d9b-8330-8ce204f17481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.659803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f39f7a3f50b3487c82db1775dc7e68a3', 'request_id': 'req-c896e6be-e976-4d9b-8330-8ce204f17481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.670740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73514440055c4024b3c84736ca27538f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.686017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73514440055c4024b3c84736ca27538f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.689076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73514440055c4024b3c84736ca27538f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.691714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73514440055c4024b3c84736ca27538f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.703515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] The token cache is being invalidate because project f39f7a3f50b3487c82db1775dc7e68a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:50.704646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.710354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.718440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 877/1763] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:50 2022] DELETE /identity/v3/projects/f39f7a3f50b3487c82db1775dc7e68a3 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:50.720525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:50.723469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:50.732865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5082f783-53ae-42cd-bfdc-4b231b674ffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73514440055c4024b3c84736ca27538f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:50.840996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 887/1764] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:50 2022] DELETE /identity/v3/users/73514440055c4024b3c84736ca27538f => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:50.845330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f7177-6b56-4582-8134-58f13c641263 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:50.846914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f7177-6b56-4582-8134-58f13c641263 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:50.848040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f7177-6b56-4582-8134-58f13c641263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:50.911231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-584f7177-6b56-4582-8134-58f13c641263 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:50.928846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c896e6be-e976-4d9b-8330-8ce204f17481 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:50.934022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-584f7177-6b56-4582-8134-58f13c641263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:51.066188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:51.066860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.067184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.067458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] PATH_INFO: `/v3/projects/6872c6e91c7c4c31999544f7bd82e864` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.069270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6872c6e91c7c4c31999544f7bd82e864)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:51.073189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:51.102239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:51.186254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6872c6e91c7c4c31999544f7bd82e864', 'request_id': 'req-499d7f14-b929-477b-b387-2637923dd00c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:51.196104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 888/1765] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:51.202951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6872c6e91c7c4c31999544f7bd82e864', 'request_id': 'req-499d7f14-b929-477b-b387-2637923dd00c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:51.206398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.207305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.207305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.233478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6872c6e91c7c4c31999544f7bd82e864', 'request_id': 'req-499d7f14-b929-477b-b387-2637923dd00c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:51.260751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] The token cache is being invalidate because project 6872c6e91c7c4c31999544f7bd82e864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:51.261130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:51.263731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-499d7f14-b929-477b-b387-2637923dd00c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:51.267603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 878/1766] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:50 2022] DELETE /identity/v3/projects/6872c6e91c7c4c31999544f7bd82e864 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:51.267926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:51.269369 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:51.276348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98196-c6f3-43a4-a72c-9b47d4d9e70d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.276645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98196-c6f3-43a4-a72c-9b47d4d9e70d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.277137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f98196-c6f3-43a4-a72c-9b47d4d9e70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.319369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-55f98196-c6f3-43a4-a72c-9b47d4d9e70d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:51.320648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-55f98196-c6f3-43a4-a72c-9b47d4d9e70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:51.480377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 889/1767] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:51.488067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:51.521702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75b20-11a1-4b88-9ddc-ce172d42e674 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:51.537248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:51.538302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.538793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.539303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.540384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:51.540939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:51.542898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:51.554914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 879/1768] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:51.579363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.579703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.579980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.607515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 890/1769] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:35:51.613927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:51.635663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc35e94-74ef-472d-9639-2c2240f037e1 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:51.638716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:51.642904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:51.648736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:51.649648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:51.650164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:51.650636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:51.651642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:51.654072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:51.709671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 880/1770] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:51.715397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:51.727963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:51.731567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f664ebfc-8070-414f-9246-2970f93e812d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:51.741003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 891/1771] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:51 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:51.742941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:51.745976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:52.052956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:52.053773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.054145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.054671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.055475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:52.055933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:52.063609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cc807-3c37-4361-b729-03b6223014ca admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:52.065596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:52.071586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:52.072830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.073282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.073707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] PATH_INFO: `/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.076962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=91c6c23d2ef64df29040fe44ff0b6a42)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:52.079948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:52.116638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 881/1772] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:35:52.117990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:52.141445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae89be-6c48-4ca1-86c3-fccbafbafa9c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:52.160929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:52.162410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.163156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.163558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.165605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:52.169698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:52.202483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91c6c23d2ef64df29040fe44ff0b6a42', 'request_id': 'req-c0aa4300-c674-4457-ac89-399b0b1ef92f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:52.231235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91c6c23d2ef64df29040fe44ff0b6a42', 'request_id': 'req-c0aa4300-c674-4457-ac89-399b0b1ef92f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:52.242694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91c6c23d2ef64df29040fe44ff0b6a42', 'request_id': 'req-c0aa4300-c674-4457-ac89-399b0b1ef92f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:52.267811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 882/1773] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:52 2022] POST /identity/v3/users => generated 289 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:52.267811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:52.293977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c88f48-c03e-4d2d-9cb2-43d53a6136b6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:52.324520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:52.333549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] The token cache is being invalidate because project 91c6c23d2ef64df29040fe44ff0b6a42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:52.335591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:52.340265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:52.348523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 892/1774] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:51 2022] DELETE /identity/v3/projects/91c6c23d2ef64df29040fe44ff0b6a42 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:52.352788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:52.358827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:52.551852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:52.553270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.556245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.556923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.560915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:52.564989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:52.704115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa4300-c674-4457-ac89-399b0b1ef92f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:52.716096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:52.716864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.717151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.717463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.718148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:52.720903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708f2d5-6d12-468b-9521-507ada05aa42 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:52.736857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 893/1775] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:52.738445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:52.738724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:52.738994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:52.788368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 883/1776] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:52 2022] POST /identity/v3/projects => generated 387 bytes in 526 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:52.799749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:52.826667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:52.833160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:52.942624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:53.033031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6870fd-8784-4416-a477-d173307b0279 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.046906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.049526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.049987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.051657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.056398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.064410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.119132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 884/1777] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:53.120679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:53.123977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:53.154817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4455aaa3-c91a-47b7-9bd4-588f8420640a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.165701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.168440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.168689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.169592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.171289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.178275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912f380e-5626-45bf-b2a5-469cab706304 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.208680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 894/1778] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:53.211385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:53.237258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 885/1779] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:53 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:53.240147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.241395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.242050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.249257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0995cf-235a-4608-918a-ae1bea47e0d0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.297244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.299330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.300019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.300695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] PATH_INFO: `/v3/projects/0815a80b3bb147ac869730c9e3663342/users/69809fded33b4419b759c360f659ae64/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.367542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0815a80b3bb147ac869730c9e3663342, user_id=69809fded33b4419b759c360f659ae64, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.368990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:53.375233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:53.386762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.393592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 886/1780] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:53.397490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:53.424520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd46d01-6b59-44d7-9449-13b48440f412 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.445639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 895/1781] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:53 2022] PUT /identity/v3/projects/0815a80b3bb147ac869730c9e3663342/users/69809fded33b4419b759c360f659ae64/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:53.454792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:53.778361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.780679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.781335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.781753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.783400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.786299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685c0f2-ccd7-4b22-bf41-f132d636274c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.792647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25641078-e44f-4ddf-bd02-81110e54b813 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.809273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.810268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.810742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.811238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.812424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:35:53.812925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.817163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 887/1782] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:53.818981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.820424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.822290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.822734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.840782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:53.886175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 896/1783] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:35:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:35:53.886175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:53.887297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] MFA Rules not processed for user `69809fded33b4419b759c360f659ae64`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:53.896424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:53.913114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd1c83-cd55-4d54-9df1-ff2fee137239 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:53.932923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:53.934154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:53.934623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:53.935114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:53.936330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:53.939605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:53.998620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 897/1784] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:53 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:54.004332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:54.062211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8fa077-1734-4f92-b4ab-17c64b99160e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:54.079577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:54.080208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.080487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.080747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.081964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:54.083644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:54.163193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 898/1785] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:53 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:54.170359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:54.201778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 888/1786] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:54.201778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:54.245226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30ca4d-8840-4d3f-8cda-13fdb7103f33 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:54.350843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:54.352677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.353270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.353453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.354790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:54.358022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:54.385064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e88004-024c-4c3c-a051-df5ee560b92c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:54.403605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 889/1787] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:54 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:54.406337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:54.407016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:54.407710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.408072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.408337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] PATH_INFO: `/v3/projects/30b24a60f887414bb72c7b3a7e5c5cb0/users/3244e0ff187d48d092c9e45771ff2a6e/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.412611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:54.417100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:54.431622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b24a60f887414bb72c7b3a7e5c5cb0, user_id=3244e0ff187d48d092c9e45771ff2a6e, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:54.434948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:54.463970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 899/1788] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:54 2022] PUT /identity/v3/projects/30b24a60f887414bb72c7b3a7e5c5cb0/users/3244e0ff187d48d092c9e45771ff2a6e/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:54.465758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:54.672169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d7e199-a26c-42a7-94d8-e1661042103e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:54.743484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e8051-902f-4bc2-a323-b12da0ed2dc4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:54.786541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:54.789445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.791579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.792155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.793691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:54.802097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b56c7a6-2607-4ec4-8b8b-f510733f3e23 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:54.834392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 900/1789] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:54.838668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.838959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.839270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.845270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:54.868049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:54.868578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:54.868725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:54.868972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:54.900439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] MFA Rules not processed for user `3244e0ff187d48d092c9e45771ff2a6e`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:54.902722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:54.994761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:54.996636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.022174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 890/1790] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:55.024100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.054021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddc245-46a3-4d06-8e79-5b35120c1c5e service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.066175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.067789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.068044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.068321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.069041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.072274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.086724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 891/1791] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:55.095103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.127743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 901/1792] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:55.133371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.136149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9b15d-1002-418e-986a-ca8b35094c29 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.171047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.172575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.173422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.174102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] PATH_INFO: `/v3/projects/f291abb1bfad4b89acb88f302d1c8112/users/41747c1c18b74513b3b7777db8013dd7/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.181441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426bb482-d455-4160-858c-d62e10c70dbe None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.204586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.205887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.206341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.206782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.207901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.230580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.232792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f291abb1bfad4b89acb88f302d1c8112, user_id=41747c1c18b74513b3b7777db8013dd7, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.241608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac68af27-2a21-47dc-8eec-6ca6020d473d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.283632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 892/1793] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:55 2022] PUT /identity/v3/projects/f291abb1bfad4b89acb88f302d1c8112/users/41747c1c18b74513b3b7777db8013dd7/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:55.299930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.300843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.301564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.323162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 902/1794] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:55 2022] POST /identity/v3/projects => generated 385 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:55.328676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.397555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] MFA Rules not processed for user `41747c1c18b74513b3b7777db8013dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:55.409178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:55.528391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:55.580942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16139e89-18a9-48d7-b88b-8d86d0d7322d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.592294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.598562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.599096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.600379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.602424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.608052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.618571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 893/1795] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:55.619859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.643221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe053a6-9eb2-4c87-a0a1-cdea69a651a3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.649438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.650083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.650363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.650713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.651324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.653453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.681905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 903/1796] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:55 2022] POST /identity/v3/users => generated 299 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:55.689361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.731123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd5a4a-f987-4292-a07d-49efdc8139eb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.731445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 894/1797] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:35:55 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:35:55.731445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.832542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:55.884426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.885667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.886423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.887394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.888771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.892928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.925492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 904/1798] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:55.933048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:55.949183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede207cf-19b4-4cfa-8e0a-d3aa287abee9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.968189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.968708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.968992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.969271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:55.969971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:55.972163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:55.983961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66457c4-0391-4a63-8994-51e916ea7e2c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:55.992121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:55.994294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:55.994294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:55.994567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] PATH_INFO: `/v3/projects/eefa24de4cb5403ca9dd966f31fccba0/users/84e107170c71432aacf2b188c5142d87/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.018834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eefa24de4cb5403ca9dd966f31fccba0, user_id=84e107170c71432aacf2b188c5142d87, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.020794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 895/1799] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:35:55 2022] POST /identity/v3/users => generated 288 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:35:56.025060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.027641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.044057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba877e-a303-4012-83b9-537d7460e4d9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.051025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 905/1800] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:55 2022] PUT /identity/v3/projects/eefa24de4cb5403ca9dd966f31fccba0/users/84e107170c71432aacf2b188c5142d87/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:56.056172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.149841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1443fc-d364-4066-bc2a-211049ab15b4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.151996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.153315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.153315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.153315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.154788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.157701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.163929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.165013 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.165514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.165936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.172454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.174727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.180320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 896/1801] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:56.181430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.193008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 906/1802] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:56.197364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28dd5c-5ae0-40c1-911e-dbce386d886f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.198556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.205902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.206847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.207260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.207638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] PATH_INFO: `/v3/projects/1021659d54874177801d497666f91a4e/users/6c34f8413b364ba7954588972046f372/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.220477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d32001-293f-42b8-9faf-ea381bda144b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.229434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021659d54874177801d497666f91a4e, user_id=6c34f8413b364ba7954588972046f372, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.230167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.231185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.231697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.232881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] PATH_INFO: `/v3/projects/eefa24de4cb5403ca9dd966f31fccba0/users/84e107170c71432aacf2b188c5142d87/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.233534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.236724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eefa24de4cb5403ca9dd966f31fccba0, user_id=84e107170c71432aacf2b188c5142d87, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.244244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e3bc4a-f184-4c3d-aa30-ab39b2e4c2d2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.270019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 897/1804] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:56 2022] PUT /identity/v3/projects/1021659d54874177801d497666f91a4e/users/6c34f8413b364ba7954588972046f372/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:56.278281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 907/1804] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:56 2022] PUT /identity/v3/projects/eefa24de4cb5403ca9dd966f31fccba0/users/84e107170c71432aacf2b188c5142d87/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:56.280241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.280423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.280562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.282185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.322395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] MFA Rules not processed for user `84e107170c71432aacf2b188c5142d87`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:56.322655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:56.380337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:56.434421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a2b9fe-e686-47ee-bd50-1d3b52816329 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.491263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:56.524336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 908/1805] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:56 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:56.544712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.545809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.546145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.547022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.547397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.548331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.550582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.566266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 898/1806] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:35:56.572796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:56.596868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744e953-a17b-4a18-ad75-47db4fa0bfa2 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.616999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.618937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.620240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.620602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] PATH_INFO: `/v3/projects/1021659d54874177801d497666f91a4e/users/6c34f8413b364ba7954588972046f372/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.625788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1021659d54874177801d497666f91a4e, user_id=6c34f8413b364ba7954588972046f372, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:56.630227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b724d-3a97-4eb4-be6e-dfd4601cd998 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:56.647174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 899/1807] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:35:56 2022] PUT /identity/v3/projects/1021659d54874177801d497666f91a4e/users/6c34f8413b364ba7954588972046f372/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:56.669087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.669557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.670241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.711120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344ca0a-648c-4ef0-985f-5443b76938bd None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:56.730822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] MFA Rules not processed for user `6c34f8413b364ba7954588972046f372`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:56.733149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:56.831483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:56.832555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:56.832945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:56.833408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:56.977908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 900/1808] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:57.029230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:57.031284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:57.046521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 909/1809] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:56 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:35:57.079644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:57.082330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:57.114149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2311fa7c-65c5-473a-ac78-970ef62a3e99 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:57.123420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:57.124480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:57.124969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:57.125462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:57.148378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:57.151170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab338fd4-26a6-4828-91d9-5600b243d93c service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:57.172074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 901/1810] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:57.357789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:57.450419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:57.602273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050e694-3476-48ac-91ab-7b9217d062ff service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:57.619854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:57.620770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:57.621068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:57.621364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] PATH_INFO: `/v3/users/3244e0ff187d48d092c9e45771ff2a6e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:57.627927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3244e0ff187d48d092c9e45771ff2a6e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:57.629716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:57.748489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3244e0ff187d48d092c9e45771ff2a6e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:57.764607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3244e0ff187d48d092c9e45771ff2a6e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:57.765556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3244e0ff187d48d092c9e45771ff2a6e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:57.766292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3244e0ff187d48d092c9e45771ff2a6e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:57.793219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-afdef3f8-ac9c-4762-bd61-3eca386dc8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3244e0ff187d48d092c9e45771ff2a6e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:57.862406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 910/1811] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:57 2022] DELETE /identity/v3/users/3244e0ff187d48d092c9e45771ff2a6e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:35:57.869252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:57.870756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:57.875843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:57.964044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:57.969780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:58.221661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 902/1812] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:35:58.232062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3f828-ef02-4dd2-b03d-68c3ac1a4bcc None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:58.233018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3f828-ef02-4dd2-b03d-68c3ac1a4bcc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:58.233834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3f828-ef02-4dd2-b03d-68c3ac1a4bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:58.254910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:58.260112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:58.392444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b4a3f828-ef02-4dd2-b03d-68c3ac1a4bcc None None] MFA Rules not processed for user `c29ce05ebc6e4672bb9557f751a48557`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:58.394246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b4a3f828-ef02-4dd2-b03d-68c3ac1a4bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:58.534690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:58.589866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cb55f-6573-4e69-bafd-2ee9b73f2e7d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:58.615606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:58.615606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:58.615606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:58.615606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:58.633259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:58.638864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ec96e-13cb-4694-86a4-05d42f380425 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:58.680794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 903/1813] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:35:58.682359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:58.683296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:58.683585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:58.690597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 911/1814] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:58.691172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 904/1815] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:35:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:35:58.703357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2c58-354b-4342-b7bf-63a1a00b533c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:58.703964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2c58-354b-4342-b7bf-63a1a00b533c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:58.704656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394f2c58-354b-4342-b7bf-63a1a00b533c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:58.802284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-394f2c58-354b-4342-b7bf-63a1a00b533c None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:58.805701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-394f2c58-354b-4342-b7bf-63a1a00b533c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:58.951416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:58.955153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:58.997971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f190ab86-9d7b-4c2c-bb73-3d093021afbc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:59.016190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:59.016715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:59.017546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.017925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.018417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] PATH_INFO: `/v3/users/41747c1c18b74513b3b7777db8013dd7` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.026463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41747c1c18b74513b3b7777db8013dd7)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:59.028796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:59.129847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 912/1816] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:59.160076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41747c1c18b74513b3b7777db8013dd7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.179946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41747c1c18b74513b3b7777db8013dd7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.202208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41747c1c18b74513b3b7777db8013dd7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.202208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41747c1c18b74513b3b7777db8013dd7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.202208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41747c1c18b74513b3b7777db8013dd7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.242450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 905/1817] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:58 2022] DELETE /identity/v3/users/41747c1c18b74513b3b7777db8013dd7 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:59.246774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.247211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.247361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.256954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:59.263742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.278869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:35:59.280577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.412542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 913/1818] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:35:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:35:59.424378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:59.425417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47840f1-9b1d-4cfc-8481-4653c8ade9e8 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:59.429709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.439982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:59.440954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.444299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdd2c8c-8aa3-473e-aeb2-d1af12691ce2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:59.449886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.450394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.451238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:59.451916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.453063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.453063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] PATH_INFO: `/v3/projects/30b24a60f887414bb72c7b3a7e5c5cb0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.453889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b24a60f887414bb72c7b3a7e5c5cb0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:59.454730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.456597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:59.490950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:59.494546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:59.507227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 906/1819] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:59.507946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:59.509369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.556605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b24a60f887414bb72c7b3a7e5c5cb0', 'request_id': 'req-9e5b3399-178f-49cf-a99d-85ecf667babc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.565988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b24a60f887414bb72c7b3a7e5c5cb0', 'request_id': 'req-9e5b3399-178f-49cf-a99d-85ecf667babc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.568648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:59.582492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b24a60f887414bb72c7b3a7e5c5cb0', 'request_id': 'req-9e5b3399-178f-49cf-a99d-85ecf667babc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.605397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] The token cache is being invalidate because project 30b24a60f887414bb72c7b3a7e5c5cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:35:59.606038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.606944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.607819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e9de3-c0b7-44fc-9037-85c88d41574c service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:59.610315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 914/1820] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:35:59 2022] DELETE /identity/v3/projects/30b24a60f887414bb72c7b3a7e5c5cb0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:35:59.618557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:35:59.621093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.671314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:59.672325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.672776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.673775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.676522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:35:59.698347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3399-178f-49cf-a99d-85ecf667babc admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:35:59.704209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:35:59.704981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:35:59.705326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:35:59.705719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] PATH_INFO: `/v3/users/84e107170c71432aacf2b188c5142d87` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:35:59.707309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e107170c71432aacf2b188c5142d87)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:59.709498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:59.716120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:35:59.772130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:35:59.773174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:35:59.778525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e107170c71432aacf2b188c5142d87'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.784662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 907/1821] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:35:59.787644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e107170c71432aacf2b188c5142d87'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.788566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e107170c71432aacf2b188c5142d87'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.789424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e107170c71432aacf2b188c5142d87'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.801368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ffa47de1-726d-44ab-9f69-a2f15be7c699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e107170c71432aacf2b188c5142d87'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:35:59.828443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 915/1822] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:35:59 2022] DELETE /identity/v3/users/84e107170c71432aacf2b188c5142d87 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:00.176843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.188039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc9e23b-030b-4c66-bf65-31e14a6d9120 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.275221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.276107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.276519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.276725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] PATH_INFO: `/v3/projects/eefa24de4cb5403ca9dd966f31fccba0` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.278206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=eefa24de4cb5403ca9dd966f31fccba0)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.280465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.333801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:00.343537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eefa24de4cb5403ca9dd966f31fccba0', 'request_id': 'req-cb3f204f-371f-4899-ae10-08d80f63a0ab'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:00.357079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eefa24de4cb5403ca9dd966f31fccba0', 'request_id': 'req-cb3f204f-371f-4899-ae10-08d80f63a0ab'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:00.368511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eefa24de4cb5403ca9dd966f31fccba0', 'request_id': 'req-cb3f204f-371f-4899-ae10-08d80f63a0ab'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:00.390238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] The token cache is being invalidate because project eefa24de4cb5403ca9dd966f31fccba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:00.391120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:00.393065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cb3f204f-371f-4899-ae10-08d80f63a0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:00.395333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 908/1823] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:00 2022] DELETE /identity/v3/projects/eefa24de4cb5403ca9dd966f31fccba0 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:00.401136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.401436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.401669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.425086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:00.426370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:00.499951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 916/1824] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:00.510531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.510967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.511374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.544471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:00.546229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:00.563256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 909/1825] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:00.566770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.578559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97100990-eb7a-41b2-bac8-21fd7ac3869c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.586079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.586734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.587019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.588005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.588721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:00.589036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.590849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.610960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 917/1826] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:00.618568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.636324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a91c5-9211-4d49-bcc9-8f859b3407a3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.644874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.645922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.646379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.647241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.649031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.653347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.715528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 910/1827] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:00 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:00.722464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.742130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.743988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:00.767988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f686c-59b1-4931-ba93-ac7fe7ae66b9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.887929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e4860b-8a06-4f42-b74b-a96d111031d6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.893354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.894180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.894549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.894837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.895548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.897705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.904562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.905234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.905510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.905801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] PATH_INFO: `/v3/projects/f291abb1bfad4b89acb88f302d1c8112` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.908494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f291abb1bfad4b89acb88f302d1c8112)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.914194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.920928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:00.928794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 918/1828] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:00 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:00.934960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:00.949384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda012b6-e84e-4978-8aa3-bf908fde2eeb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:00.957094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:00.957752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:00.958039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:00.958317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:00.960418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:00.962913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:00.985031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 919/1829] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:01.000393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.004152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f291abb1bfad4b89acb88f302d1c8112', 'request_id': 'req-7c00b4bc-82ef-41af-9878-863da3fb38d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.008540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a80927-bd3c-4c3b-be7b-c7b82afeeb37 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.012566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f291abb1bfad4b89acb88f302d1c8112', 'request_id': 'req-7c00b4bc-82ef-41af-9878-863da3fb38d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.024803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f291abb1bfad4b89acb88f302d1c8112', 'request_id': 'req-7c00b4bc-82ef-41af-9878-863da3fb38d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.048820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] The token cache is being invalidate because project f291abb1bfad4b89acb88f302d1c8112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:01.050431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.051485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.054532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 911/1830] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:00 2022] DELETE /identity/v3/projects/f291abb1bfad4b89acb88f302d1c8112 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:01.061730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.063548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:01.109154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.110891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.111453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.111964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.113214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.118319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.127622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:01.143322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 920/1831] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:00 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:01.151405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.155406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:01.188106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c00b4bc-82ef-41af-9878-863da3fb38d6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.193762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.194605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.194975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.195477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] PATH_INFO: `/v3/users/6c34f8413b364ba7954588972046f372` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.197135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c34f8413b364ba7954588972046f372)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.199487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.202722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5c982-eebf-438b-8db7-d408994f9325 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.212850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.214664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.215402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.216004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] PATH_INFO: `/v3/projects/a26e6677a6234a90be68d9b89b04c37e/users/fd08d334a02f48da9805bd4f83934df5/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.254840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a26e6677a6234a90be68d9b89b04c37e, user_id=fd08d334a02f48da9805bd4f83934df5, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.261797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9354db-4ffa-4619-87e4-1dc87808945b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.273263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c34f8413b364ba7954588972046f372'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.283754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c34f8413b364ba7954588972046f372'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.288170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 921/1832] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:01 2022] PUT /identity/v3/projects/a26e6677a6234a90be68d9b89b04c37e/users/fd08d334a02f48da9805bd4f83934df5/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:01.294521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c34f8413b364ba7954588972046f372'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.295546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c34f8413b364ba7954588972046f372'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.296173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c34f8413b364ba7954588972046f372'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:01.303833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.304172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.304515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.324102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 912/1833] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:01 2022] DELETE /identity/v3/users/6c34f8413b364ba7954588972046f372 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:01.353507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] MFA Rules not processed for user `fd08d334a02f48da9805bd4f83934df5`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:01.359184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:01.539576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 922/1834] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:01.540946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.542561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:01.631124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6da46-1315-4e02-a547-b6778a669042 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.635983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.636630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.636907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.637178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.637837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.639922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.680316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 913/1835] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:01 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:01.681530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.702728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a52a8-4ecd-4f9e-9c7f-daa4eb185e2c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.828144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:01.851970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.853860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.854660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.855476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.856950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.861882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.911745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 923/1836] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:01 2022] POST /identity/v3/users => generated 299 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:01.915680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.926591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2fdec-abba-442d-b071-ec132f46a1e6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:01.936809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:01.937542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:01.937895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:01.938270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:01.939580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:01.951818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:01.971137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 914/1837] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:01.975448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:01.996111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8614e3-bf2b-445a-abb7-d6fcfbd5d302 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.008615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.010349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.010933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.011279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] PATH_INFO: `/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3/users/3afed6f61b7e4ebf9188d0f91b307712/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.058322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df835b4e97a4aba9e43585a2ddfd9f3, user_id=3afed6f61b7e4ebf9188d0f91b307712, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.061880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.097550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 924/1838] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:01 2022] PUT /identity/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3/users/3afed6f61b7e4ebf9188d0f91b307712/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:02.098912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.113753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e402e5f0-e7c4-4b4e-9233-5475f1e12cd1 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.192191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:02.343058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.344147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.344690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.345229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.346015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.347856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.386006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.390162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:02.399220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 915/1839] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:36:02.407247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.427939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb97d8-35f0-4dcd-b344-eaab0daa4f63 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.433998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf0430-9917-4b2a-a839-0473f9598e4f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.442278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.443148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.443525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.443868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] PATH_INFO: `/v3/projects/1021659d54874177801d497666f91a4e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.445987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1021659d54874177801d497666f91a4e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.447694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.448504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.448799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.449038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] PATH_INFO: `/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3/users/3afed6f61b7e4ebf9188d0f91b307712/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.450762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.457119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2df835b4e97a4aba9e43585a2ddfd9f3, user_id=3afed6f61b7e4ebf9188d0f91b307712, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.463859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab00e0-1793-4afb-b838-c1eb22b8307c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.522090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 916/1840] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:02 2022] PUT /identity/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3/users/3afed6f61b7e4ebf9188d0f91b307712/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:02.544107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.544879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.545808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.609636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1021659d54874177801d497666f91a4e', 'request_id': 'req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:02.622352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1021659d54874177801d497666f91a4e', 'request_id': 'req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:02.636292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1021659d54874177801d497666f91a4e', 'request_id': 'req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:02.645564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] MFA Rules not processed for user `3afed6f61b7e4ebf9188d0f91b307712`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:02.648463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:02.657779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] The token cache is being invalidate because project 1021659d54874177801d497666f91a4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:02.658197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:02.659339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-1eaf4e4a-0e34-41db-bc41-f95d1bb73b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:02.665550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 925/1841] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:02 2022] DELETE /identity/v3/projects/1021659d54874177801d497666f91a4e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:02.667707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15e00-620b-4f02-8c22-bdad7f03864a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.667998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15e00-620b-4f02-8c22-bdad7f03864a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.668282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e15e00-620b-4f02-8c22-bdad7f03864a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.702141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a7e15e00-620b-4f02-8c22-bdad7f03864a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:02.703799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a7e15e00-620b-4f02-8c22-bdad7f03864a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:02.795413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 926/1842] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:02.805365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.805931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.806802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.823163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 917/1843] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:02.841531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.843646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:02.844751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:02.845199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:02.855881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 927/1844] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:02.862244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.879557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a027dda-5eea-4f8d-ae59-eea283e704de None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.887025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.887650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.887948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.888197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.888923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:02.889231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.891001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.908979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 928/1845] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:36:02.913764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:02.925539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0753f84e-795c-4342-82ba-e84d371b6106 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:02.933074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:02.933717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:02.933990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:02.934283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:02.934930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:02.937147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:02.965420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:02.978149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 929/1846] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:02 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:36:02.994519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.023755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b481ed4-0ff3-4bb3-9dca-d1196c06183b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.091228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:03.098112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.099489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.100345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.102964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.129397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eff4aa-983c-441a-9de4-d7c08c2c8c79 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.138081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.138724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.139008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.139328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.142888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.142888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.177267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 930/1847] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:02 2022] POST /identity/v3/users => generated 282 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:03.184391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.204054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038d0c8-8c11-4c01-8887-cf5b4189a627 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.212689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.213720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.214123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.214497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.215306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.217484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.236398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 931/1848] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:03.237430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.249945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6917e-d55a-44c9-a9a2-a51cec0e9ae0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.264300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.264946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.265227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.265492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.267147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.268298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f9895f-fbec-450f-a812-49e1fccfd910 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.285194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 932/1849] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:03.286925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1a60-0369-4131-9a7b-0c043333513b None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.287240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1a60-0369-4131-9a7b-0c043333513b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.287610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f1a60-0369-4131-9a7b-0c043333513b None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.289012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 933/1850] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:36:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:36:03.291359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1a60-0369-4131-9a7b-0c043333513b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.296695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.298298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e639b8f-a395-4087-9cf1-9f869f879119 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.314155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f1a60-0369-4131-9a7b-0c043333513b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.319510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.320303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.320640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.320985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] PATH_INFO: `/v3/projects/a981ce44ab3e487a9a50b0cd16349ec2/users/7f19cbb6704546148580f173a33695f6/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.327012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 918/1851] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:03.347198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a981ce44ab3e487a9a50b0cd16349ec2, user_id=7f19cbb6704546148580f173a33695f6, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.350214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e1b512-8665-4ec1-b6a5-99cb71317f6e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.356570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.356816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.357061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.371962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 934/1852] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:03 2022] PUT /identity/v3/projects/a981ce44ab3e487a9a50b0cd16349ec2/users/7f19cbb6704546148580f173a33695f6/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:03.395401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.396060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.396348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.411235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:03.413014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:03.435695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] MFA Rules not processed for user `7f19cbb6704546148580f173a33695f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:03.437730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:03.526915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 935/1853] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:03.527967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.529439 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:03.531515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 919/1854] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:03.533495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.590137 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:03.634408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231c3674-054c-41e2-a881-1c8f40b2e74e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.635453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9b950-2c50-4b9a-8f8c-3adb1443e87c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.639842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.640315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.640588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.640870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.641144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] PATH_INFO: `/v3/users/fd08d334a02f48da9805bd4f83934df5` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.641236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.641535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.641937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.642530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd08d334a02f48da9805bd4f83934df5)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.642647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.644593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.644773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.680590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 920/1855] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:03 2022] POST /identity/v3/projects => generated 365 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:36:03.686215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.711739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:03.720578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd08d334a02f48da9805bd4f83934df5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:03.730764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd08d334a02f48da9805bd4f83934df5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:03.731582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd08d334a02f48da9805bd4f83934df5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:03.732697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd08d334a02f48da9805bd4f83934df5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:03.774957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e549d7-67a5-4d30-a600-cf6b1d9bdc2b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.778854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd08d334a02f48da9805bd4f83934df5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:03.780944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.781647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.781965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.782334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.783209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.785706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.813398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 921/1856] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:03 2022] POST /identity/v3/users => generated 289 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:03.814480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.817180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:03.825305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 936/1857] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:03 2022] DELETE /identity/v3/users/fd08d334a02f48da9805bd4f83934df5 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:03.827236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:03.829620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:03.917177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9d75-433c-495b-8f91-47529df3cb70 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.925248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95418c47-5179-4cb2-b640-6335cb59792c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:03.934787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:03.935878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.936325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.936785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:03.938388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:03.941584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54137e57-7dc4-400c-8dc7-52cc03128557 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:03.965647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 937/1858] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:03.965647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:03.965647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:03.966124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.002018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:04.003173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.003819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.004100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.004253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.004485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.006222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.017602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 938/1859] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:04.019334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.023970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.026647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.042796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93974e14-498e-44a0-a07f-6affee41e8db None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.049715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 922/1860] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:04.049715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.051848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.055220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.056783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.057484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.058127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] PATH_INFO: `/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff/users/e93a982e68da465d88ec48176d718078/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.092296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] RBAC: Authorizing `identity:create_grant(project_id=633e4023f6094c02b9df9d30a4c9a7ff, user_id=e93a982e68da465d88ec48176d718078, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.101308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.120097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:04.135669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 939/1861] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:04 2022] PUT /identity/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff/users/e93a982e68da465d88ec48176d718078/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:04.140882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.171411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87fd607-bbfc-43d0-a64c-445f87ff30a1 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.179243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:04.276625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.277679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.278168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.278652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.293081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebbf24-65a6-4c52-9e8e-d11c1672899d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.298398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.301250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.304444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.307141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.307141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.307141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.308046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.310635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9130fff6-8e8d-4ade-b6f2-eaeff98a70a4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.329386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 923/1862] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:04.336173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 940/1863] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:04.342703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.359266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247fe60c-b0aa-4ef8-9f9a-b613f70b1e51 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.379946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.382347 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.383025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.384880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] PATH_INFO: `/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff/users/e93a982e68da465d88ec48176d718078/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.388824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=633e4023f6094c02b9df9d30a4c9a7ff, user_id=e93a982e68da465d88ec48176d718078, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.392733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.411950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 924/1864] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:04 2022] PUT /identity/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff/users/e93a982e68da465d88ec48176d718078/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:04.424354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.424672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.425010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.463275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] MFA Rules not processed for user `e93a982e68da465d88ec48176d718078`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:04.464667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.591599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 941/1865] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:04.607499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.610540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.715863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1488-6beb-464b-9c64-0f676b58e7eb admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.720653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.721438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.721862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.722238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.739026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:04.741777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:04.758062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 925/1866] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:04.896511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.899647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:04.929497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:04.947911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7d8eb3-e357-4486-bc38-e8f0a38028ca service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:04.956017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:04.957479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:04.958263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:04.958929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:04.973564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:05.067402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7b0ca-a98f-4499-be8c-4cfd3601c289 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:05.077086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:05.078133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:05.078622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:05.079169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] PATH_INFO: `/v3/projects/a26e6677a6234a90be68d9b89b04c37e` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:05.081331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a26e6677a6234a90be68d9b89b04c37e)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:05.084271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:05.109796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:05.112896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:05.135485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 926/1867] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:05.180683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:05.214815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a26e6677a6234a90be68d9b89b04c37e', 'request_id': 'req-9c773160-c678-444f-b9ee-13186c0c82d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.236204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a26e6677a6234a90be68d9b89b04c37e', 'request_id': 'req-9c773160-c678-444f-b9ee-13186c0c82d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.253826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a26e6677a6234a90be68d9b89b04c37e', 'request_id': 'req-9c773160-c678-444f-b9ee-13186c0c82d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.279883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] The token cache is being invalidate because project a26e6677a6234a90be68d9b89b04c37e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:05.280399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.281569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.284454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 942/1868] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:04 2022] DELETE /identity/v3/projects/a26e6677a6234a90be68d9b89b04c37e => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:05.289500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:05.313809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b01fe-2bc8-4d68-874a-5b7d50b09743 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:05.348031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:05.484532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:05.486004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:05.486530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:05.487057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] PATH_INFO: `/v3/users/3afed6f61b7e4ebf9188d0f91b307712` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:05.489507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3afed6f61b7e4ebf9188d0f91b307712)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:05.493657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:05.593037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3afed6f61b7e4ebf9188d0f91b307712'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.613572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3afed6f61b7e4ebf9188d0f91b307712'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.629084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3afed6f61b7e4ebf9188d0f91b307712'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.630090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3afed6f61b7e4ebf9188d0f91b307712'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.631213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0f42a461-b767-4fe8-9aff-cccd1b8db3cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3afed6f61b7e4ebf9188d0f91b307712'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:05.657943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 927/1869] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:05 2022] DELETE /identity/v3/users/3afed6f61b7e4ebf9188d0f91b307712 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:06.066961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.068787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:06.088725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c773160-c678-444f-b9ee-13186c0c82d7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.172520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.173437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.173750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.174121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] PATH_INFO: `/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.175638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2df835b4e97a4aba9e43585a2ddfd9f3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.177732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.240758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2df835b4e97a4aba9e43585a2ddfd9f3', 'request_id': 'req-15a2e806-752c-4294-a9d3-2405116433a9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:06.248715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2df835b4e97a4aba9e43585a2ddfd9f3', 'request_id': 'req-15a2e806-752c-4294-a9d3-2405116433a9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:06.259759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2df835b4e97a4aba9e43585a2ddfd9f3', 'request_id': 'req-15a2e806-752c-4294-a9d3-2405116433a9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:06.281230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] The token cache is being invalidate because project 2df835b4e97a4aba9e43585a2ddfd9f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:06.282196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:06.283848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-15a2e806-752c-4294-a9d3-2405116433a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:06.287497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 943/1870] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:06 2022] DELETE /identity/v3/projects/2df835b4e97a4aba9e43585a2ddfd9f3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:06.295614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.296313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.296799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.337368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:06.340172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:06.452959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 928/1871] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:06.463329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.463783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.464247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.501740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:06.504861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:06.525203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 944/1872] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:06.531559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.550284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4277c95-f578-4a25-a512-899f1f131a84 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.576348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.579531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.580095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.580660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.582159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:06.582737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.586593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.620274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 929/1873] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:06.633480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.646494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be4252-abc7-4fc5-926f-f70ddc6f78cb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.652249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.652887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.653230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.653717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.654548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.656940 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.698967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 945/1874] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:06 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:06.700091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.717419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25522413-a00a-42ef-845b-ffc03c719872 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.748897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:06.811444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.812339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.812749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.813122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.813994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.816150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.852242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 930/1875] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:06 2022] POST /identity/v3/users => generated 291 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:06.859373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.877597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8cc3-24ea-40f2-a9e8-f11cf2084fd2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.886106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.887189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.887664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.888236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.889493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.892598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.915171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 946/1876] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:06.920101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.939283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9e130-33c9-4641-a18a-c678fc913c54 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.946216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.947198 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.947573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.948103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:06.949039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:06.950860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0450a-3642-43b1-b7c8-6a052e0f689d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:06.965379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 931/1877] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:06.970224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:06.986785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee19ed-40d9-465d-8410-b1be58effe0a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:06.994090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:06.995186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:06.995651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:06.996190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] PATH_INFO: `/v3/projects/63247ffeaf1d46af8397df460cad4235/users/2c60c01b0c454b8ebf1b6b98ffd42636/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.020332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63247ffeaf1d46af8397df460cad4235, user_id=2c60c01b0c454b8ebf1b6b98ffd42636, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.024041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.027657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:07.051599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 947/1878] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:06 2022] PUT /identity/v3/projects/63247ffeaf1d46af8397df460cad4235/users/2c60c01b0c454b8ebf1b6b98ffd42636/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:07.064676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.065105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.065511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.110660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] MFA Rules not processed for user `2c60c01b0c454b8ebf1b6b98ffd42636`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:07.112027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:07.206007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 932/1879] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:07.211431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.285132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e11745-3026-43b4-8b17-9269e3a65273 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.290034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.290672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.290950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.291268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.291917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.293911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.332732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 948/1880] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:07 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:36:07.338779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.359861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e963e-ba8e-4764-9bb1-9acd0486c3e1 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.433841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.434479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.434914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.435230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.435882 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.437850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.470515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 933/1881] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:07 2022] POST /identity/v3/users => generated 297 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:07.476557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.490436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff925c0-e3d0-4ef8-98eb-bd0778d1ef58 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.497078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.497915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.498235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.498578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.499485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.501421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.514810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 949/1882] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:07.521350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.533517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2802b1-85a9-40e1-b395-4b620ba70407 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.538818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.539580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.539843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.540100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] PATH_INFO: `/v3/projects/66f26fdb78c648b29fc821402b3c8955/users/14d78fcfe84242878fb806ac0a753074/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.569048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f26fdb78c648b29fc821402b3c8955, user_id=14d78fcfe84242878fb806ac0a753074, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.574979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.579245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:07.602646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 934/1883] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:07 2022] PUT /identity/v3/projects/66f26fdb78c648b29fc821402b3c8955/users/14d78fcfe84242878fb806ac0a753074/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:07.607556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.626215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59faac85-dacc-4072-a8c1-094c20c87de6 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.756029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.756995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.757396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.757790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.758802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.760939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f28ad0-8f8e-44e4-831b-f476fecd0201 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.783134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 950/1884] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:07 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:07.791929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:07.815801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda18612-b865-46a9-bc32-c6af2f89f1e9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:07.828225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:07.830000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.830587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.831184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] PATH_INFO: `/v3/projects/66f26fdb78c648b29fc821402b3c8955/users/14d78fcfe84242878fb806ac0a753074/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.835209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f26fdb78c648b29fc821402b3c8955, user_id=14d78fcfe84242878fb806ac0a753074, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:07.840928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:07.862988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 935/1885] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:07 2022] PUT /identity/v3/projects/66f26fdb78c648b29fc821402b3c8955/users/14d78fcfe84242878fb806ac0a753074/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:07.880222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:07.880517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:07.880755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:07.936693 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] MFA Rules not processed for user `14d78fcfe84242878fb806ac0a753074`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:07.938343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:08.113967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:08.163824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 951/1886] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:08.307105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:08.310512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:08.501067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccec33b-ae7a-40ca-bbb4-ef03c9b026bf admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:08.510756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:08.512754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:08.512754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:08.512754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:08.547427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:08.547427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:08.548115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 936/1887] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:08.747704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:08.760163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204abe7-1b63-46d2-be39-56dcabcdaac1 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:08.765426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:08.766352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:08.766901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:08.767419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:08.863507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:08.865320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5db878-f6f9-4b0b-988d-0b42668b0255 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:08.880490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 952/1888] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:09.379770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:09.501762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b0bf2c-217c-4879-8e73-6fc44c49b8fc service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:09.514144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:09.515248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:09.515698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:09.516144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] PATH_INFO: `/v3/users/2c60c01b0c454b8ebf1b6b98ffd42636` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:09.518295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c60c01b0c454b8ebf1b6b98ffd42636)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:09.522004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:09.644929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c60c01b0c454b8ebf1b6b98ffd42636'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:09.659303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c60c01b0c454b8ebf1b6b98ffd42636'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:09.673485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c60c01b0c454b8ebf1b6b98ffd42636'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:09.674589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c60c01b0c454b8ebf1b6b98ffd42636'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:09.675744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c60c01b0c454b8ebf1b6b98ffd42636'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:09.708549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 937/1889] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:09 2022] DELETE /identity/v3/users/2c60c01b0c454b8ebf1b6b98ffd42636 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:09.713776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:09.714321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:09.714816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:09.745912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:09.747936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:09.864705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 953/1890] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:09.884021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:09.891245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:09.916880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:10.013995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262d08c-46d2-49af-b174-94482414148b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:10.021686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:10.022816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:10.023340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:10.024614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:10.048352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:10.051558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:10.074554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 938/1891] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:10.710880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:10.713210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:10.758393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92456a97-6a00-4194-8b27-afdc796eadc6 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:10.780846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:10.783651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:10.784289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:10.785883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] PATH_INFO: `/v3/projects/63247ffeaf1d46af8397df460cad4235` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:10.803792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63247ffeaf1d46af8397df460cad4235)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:10.806612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:10.917332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63247ffeaf1d46af8397df460cad4235', 'request_id': 'req-82963e1f-e8ad-4675-bfd2-f555705b30c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:10.943727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63247ffeaf1d46af8397df460cad4235', 'request_id': 'req-82963e1f-e8ad-4675-bfd2-f555705b30c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:10.957673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63247ffeaf1d46af8397df460cad4235', 'request_id': 'req-82963e1f-e8ad-4675-bfd2-f555705b30c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:10.972486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] The token cache is being invalidate because project 63247ffeaf1d46af8397df460cad4235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:10.972992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:10.973978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:10.977234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 954/1892] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:10 2022] DELETE /identity/v3/projects/63247ffeaf1d46af8397df460cad4235 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:10.986626 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:11.007838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c01e-c81c-4bfa-b662-6735c20d8f5e service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:11.155328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:11.157560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:11.158045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:11.158512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] PATH_INFO: `/v3/users/14d78fcfe84242878fb806ac0a753074` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:11.160706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14d78fcfe84242878fb806ac0a753074)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:11.165094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:11.266851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14d78fcfe84242878fb806ac0a753074'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:11.276770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14d78fcfe84242878fb806ac0a753074'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:11.289847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14d78fcfe84242878fb806ac0a753074'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:11.291012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14d78fcfe84242878fb806ac0a753074'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:11.291892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a0cd6283-25e2-4bb2-b1a2-94bf8fee9db1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14d78fcfe84242878fb806ac0a753074'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:11.354757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 939/1893] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:10 2022] DELETE /identity/v3/users/14d78fcfe84242878fb806ac0a753074 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:11.784935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:11.788017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:11.810144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82963e1f-e8ad-4675-bfd2-f555705b30c7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:11.829670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:11.934309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:11.935527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:11.935979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:11.936430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] PATH_INFO: `/v3/projects/66f26fdb78c648b29fc821402b3c8955` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:11.938379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f26fdb78c648b29fc821402b3c8955)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:11.941232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.043569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f26fdb78c648b29fc821402b3c8955', 'request_id': 'req-28ef24a1-b488-4882-ad59-a9747bbf3d32'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:12.054196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f26fdb78c648b29fc821402b3c8955', 'request_id': 'req-28ef24a1-b488-4882-ad59-a9747bbf3d32'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:12.068312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f26fdb78c648b29fc821402b3c8955', 'request_id': 'req-28ef24a1-b488-4882-ad59-a9747bbf3d32'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:12.094530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] The token cache is being invalidate because project 66f26fdb78c648b29fc821402b3c8955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:12.095549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:12.096957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-28ef24a1-b488-4882-ad59-a9747bbf3d32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:12.099858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 955/1894] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:11 2022] DELETE /identity/v3/projects/66f26fdb78c648b29fc821402b3c8955 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:12.107154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.107972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.107972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.132960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:12.134055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:12.167792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:12.227697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 940/1895] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:12.236995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.237321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.237531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.261199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:12.262666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:12.273250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 956/1896] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:12.279986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.291919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd22eca-035b-4a81-baad-eee3ccaba9cb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.297765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.298428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.298691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.298991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.299748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:12.300070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.301735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.316061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 941/1897] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:36:12.321064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.332712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7355d3c1-2008-4a5d-b0f3-412c33186db0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.338661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.339577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.339977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.340385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.341267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.343994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.393361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 957/1898] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:12 2022] POST /identity/v3/projects => generated 385 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:12.398339 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.411032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5058b434-58e7-4d47-bab0-6ba5ca5a6124 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.478689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.479350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.479673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.479987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.480730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.482794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.508852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 942/1899] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:12 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:36:12.513702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.525163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4be61f4-8bf4-4f93-b1ce-780a8ad7e1de admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.530373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.531006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.531313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.531581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.532313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.534022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.547007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 958/1900] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:12.551828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.563536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebf33e-a184-4abb-ad5b-1b37cd0245f7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.568581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.569220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.569500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.569773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.570437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.572160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b925c-7f92-4cec-be4c-fb3e83581226 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.585037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 943/1901] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:12.589707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.602081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28db8-7718-40b7-9b08-2290c317c559 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.608799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.609266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.609543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.609820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] PATH_INFO: `/v3/projects/8c4df9c4f14e4e44b0b66e2d905fd8c7/users/f421c9750b7e4998a06f93686211d3c6/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.642958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c4df9c4f14e4e44b0b66e2d905fd8c7, user_id=f421c9750b7e4998a06f93686211d3c6, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.645789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.671636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 959/1902] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:12 2022] PUT /identity/v3/projects/8c4df9c4f14e4e44b0b66e2d905fd8c7/users/f421c9750b7e4998a06f93686211d3c6/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:12.689562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.690048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.690634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.746001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] MFA Rules not processed for user `f421c9750b7e4998a06f93686211d3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:12.748132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:12.854094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 944/1903] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:12.869734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:12.872002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:12.932385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:12.954838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426caf15-dd6f-4933-9086-66a01e933bfc admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:12.958862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:12.959548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:12.959819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:12.960084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:12.970602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:12.972121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25ebf7c-1771-4655-83ab-a74cb12bed3f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:12.983416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 960/1904] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:22.850014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:22.977845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=9 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:28.256160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:28.257784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:28.337781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b2671e-2102-499f-9799-9a43f4010359 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:28.348749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:28.351823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:28.353324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:28.353324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] PATH_INFO: `/v3/users/7f19cbb6704546148580f173a33695f6` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:28.355480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f19cbb6704546148580f173a33695f6)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:28.360510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:28.463283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f19cbb6704546148580f173a33695f6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:28.479152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f19cbb6704546148580f173a33695f6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:28.494909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f19cbb6704546148580f173a33695f6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:28.497964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f19cbb6704546148580f173a33695f6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:28.499017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f19cbb6704546148580f173a33695f6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:28.531051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 945/1905] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:28 2022] DELETE /identity/v3/users/7f19cbb6704546148580f173a33695f6 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:28.536557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:28.537126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:28.537267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:28.562358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:28.564766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:28.702454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 961/1906] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:28.725204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:28.727417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:28.803619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871cb86-310d-433d-968a-7e7980b6b9b4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:28.809431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:28.810519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:28.810998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:28.811504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:28.828419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:28.831350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:28.847111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 946/1907] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:29.377654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:29.382164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:29.412977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd8a0ea-2601-449b-ab17-49667146f9fc None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:29.422856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:29.424646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:29.425095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:29.425490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] PATH_INFO: `/v3/users/69809fded33b4419b759c360f659ae64` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:29.429608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] RBAC: Authorizing `identity:delete_user(user_id=69809fded33b4419b759c360f659ae64)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:29.436712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:29.601984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:29.609991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:29.612329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69809fded33b4419b759c360f659ae64'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.640559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765aee7f-c134-4d9e-976a-8cfe32b04dec service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:29.651441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69809fded33b4419b759c360f659ae64'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.654223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69809fded33b4419b759c360f659ae64'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.654464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:29.654952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:29.655556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:29.655556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] PATH_INFO: `/v3/projects/a981ce44ab3e487a9a50b0cd16349ec2` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:29.658956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69809fded33b4419b759c360f659ae64'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.659247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a981ce44ab3e487a9a50b0cd16349ec2)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:29.661060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:29.687901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bdcce42d-7237-4c60-8696-db1e0434878b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69809fded33b4419b759c360f659ae64'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.725938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:29.760308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 962/1908] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:29 2022] DELETE /identity/v3/users/69809fded33b4419b759c360f659ae64 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:29.762315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:29.762782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:29.763273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:29.790437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:29.793310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:29.813658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a981ce44ab3e487a9a50b0cd16349ec2', 'request_id': 'req-4b77dad7-6a29-404f-90a1-645b0ce1da01'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.824489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a981ce44ab3e487a9a50b0cd16349ec2', 'request_id': 'req-4b77dad7-6a29-404f-90a1-645b0ce1da01'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.852048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a981ce44ab3e487a9a50b0cd16349ec2', 'request_id': 'req-4b77dad7-6a29-404f-90a1-645b0ce1da01'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.868718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] The token cache is being invalidate because project a981ce44ab3e487a9a50b0cd16349ec2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:29.869046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.870016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:29.872845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 947/1909] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:29 2022] DELETE /identity/v3/projects/a981ce44ab3e487a9a50b0cd16349ec2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:29.878440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:29.883469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:29.960943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 963/1910] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:29.973832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:29.975722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:29.998789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:30.041872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b77dad7-6a29-404f-90a1-645b0ce1da01 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:30.054423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:30.056205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:30.056723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:30.057254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] PATH_INFO: `/v3/users/e93a982e68da465d88ec48176d718078` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:30.059969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93a982e68da465d88ec48176d718078)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:30.066551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:30.067246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf615c-1475-41f8-9632-f037a79c822a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:30.073041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:30.073677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:30.073954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:30.074280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:30.088792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:30.090262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:30.103825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 964/1911] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:30.162105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93a982e68da465d88ec48176d718078'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.173678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93a982e68da465d88ec48176d718078'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.187557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93a982e68da465d88ec48176d718078'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.188700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93a982e68da465d88ec48176d718078'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.189456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3b1e6097-9963-4bb7-a740-26ae438036ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93a982e68da465d88ec48176d718078'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.220264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 948/1912] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:29 2022] DELETE /identity/v3/users/e93a982e68da465d88ec48176d718078 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:30.235285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:30.237044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:30.260087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2910a1b-1e71-4e02-85c5-67b36cf08a1e service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:30.461340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:30.463519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:30.464346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:30.465122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] PATH_INFO: `/v3/users/f421c9750b7e4998a06f93686211d3c6` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:30.470556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f421c9750b7e4998a06f93686211d3c6)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:30.475668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:30.572816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f421c9750b7e4998a06f93686211d3c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.592425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f421c9750b7e4998a06f93686211d3c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.595185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f421c9750b7e4998a06f93686211d3c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.596773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f421c9750b7e4998a06f93686211d3c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.618719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f421c9750b7e4998a06f93686211d3c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:30.672447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 965/1913] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:30 2022] DELETE /identity/v3/users/f421c9750b7e4998a06f93686211d3c6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:30.674467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:30.674467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:30.675312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:30.696845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:30.698174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:30.739815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:30.769261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:30.771365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:30.820181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 949/1914] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:30.832977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:30.842343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:30.899679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:30.959307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0cf77d-4d44-4bde-b93f-525f3fa7a1a2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:30.965172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:30.965874 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:30.966376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:30.966826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] PATH_INFO: `/v3/projects/0815a80b3bb147ac869730c9e3663342` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:30.968884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0815a80b3bb147ac869730c9e3663342)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:30.971382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.005763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44208c3d-0d31-42c2-8801-b1bc0c547498 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.013485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.015401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.015965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.016442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.047784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.052387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.066503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0815a80b3bb147ac869730c9e3663342', 'request_id': 'req-a5d4dee1-3abd-4711-9096-6531e170f959'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.076160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0815a80b3bb147ac869730c9e3663342', 'request_id': 'req-a5d4dee1-3abd-4711-9096-6531e170f959'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.081513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 950/1915] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:31.095002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0815a80b3bb147ac869730c9e3663342', 'request_id': 'req-a5d4dee1-3abd-4711-9096-6531e170f959'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.110989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.117514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.168705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf5e32-c3af-45d3-8ce7-20a0437c49a8 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.184044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] The token cache is being invalidate because project 0815a80b3bb147ac869730c9e3663342 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:31.187396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.190376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a5d4dee1-3abd-4711-9096-6531e170f959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.192681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 966/1916] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:30 2022] DELETE /identity/v3/projects/0815a80b3bb147ac869730c9e3663342 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:31.200083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1438-24f9-4805-b6e7-72515f737ead None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.200444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1438-24f9-4805-b6e7-72515f737ead None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.200749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf1438-24f9-4805-b6e7-72515f737ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.266752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e9cf1438-24f9-4805-b6e7-72515f737ead None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:31.271014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e9cf1438-24f9-4805-b6e7-72515f737ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.291619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.292322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.293282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.293573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] PATH_INFO: `/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.296031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=633e4023f6094c02b9df9d30a4c9a7ff)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.302113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.398343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 967/1917] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:31.400861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '633e4023f6094c02b9df9d30a4c9a7ff', 'request_id': 'req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.413048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fda1b-f271-44fb-aa8d-f0877dcfef8f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.413471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fda1b-f271-44fb-aa8d-f0877dcfef8f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.413956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fda1b-f271-44fb-aa8d-f0877dcfef8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.417485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '633e4023f6094c02b9df9d30a4c9a7ff', 'request_id': 'req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.432643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '633e4023f6094c02b9df9d30a4c9a7ff', 'request_id': 'req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.442927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:31.452240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-595fda1b-f271-44fb-aa8d-f0877dcfef8f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:31.454364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-595fda1b-f271-44fb-aa8d-f0877dcfef8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.454550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] The token cache is being invalidate because project 633e4023f6094c02b9df9d30a4c9a7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:31.454714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.455622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81a8259a-6610-4c7a-bb2b-f04b4aff5f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:31.458853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 951/1918] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:31 2022] DELETE /identity/v3/projects/633e4023f6094c02b9df9d30a4c9a7ff => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:31.467158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.467471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.467729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.504016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:31.506250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.604161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 952/1919] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:31.614591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 968/1920] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:31.617318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.618036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.618335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.618597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.634986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:31.636071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1773f9-9c6c-4f45-97d3-cb373698f681 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.642738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:31.644815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.645087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.645863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.646360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.646852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.648181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:31.648684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.651899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.660211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 969/1921] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:31.666054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.678184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c0a5-622a-42da-b1cb-a79e01a07f98 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.681969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 953/1922] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:36:31.685089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.686039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.686536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.687026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.688111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:31.688350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.688640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.690315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.705154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 970/1923] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:31.710582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.722531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8b0a36-8c64-491a-a4e8-a21ec7bb6fc5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.724865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a43fa2-b09f-4e99-a330-f59d9029a72c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.730244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.731150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.731446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.731867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:31.732086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.733203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:31.733455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.733727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:31.734602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:31.735941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.736144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:31.740185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:31.786296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 971/1924] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:31.788169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.791508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:31.807065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 954/1925] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:31.808916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:31.950550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c08714-ac83-47d1-88dd-1ccd62ca27b2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:31.988396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a20cd9-e732-44aa-8be6-0292fb565651 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.000127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.001111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.001580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.002029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.003340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.007423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.051400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 955/1926] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:31 2022] POST /identity/v3/users => generated 282 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:32.053481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.085609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07a55a7-662f-4ffd-8d75-dfaa4fa57097 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.090047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.090974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.091270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.091606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] PATH_INFO: `/v3/projects/8c4df9c4f14e4e44b0b66e2d905fd8c7` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.093134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c4df9c4f14e4e44b0b66e2d905fd8c7)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.096973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.097991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.099637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.100915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.101639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.104103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.109111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.149494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 956/1927] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:32.149494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.160226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353d2987-8ddd-4423-83f2-bbaccee58c47 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.188716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c4df9c4f14e4e44b0b66e2d905fd8c7', 'request_id': 'req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:32.199903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c4df9c4f14e4e44b0b66e2d905fd8c7', 'request_id': 'req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:32.209039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:32.211847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c4df9c4f14e4e44b0b66e2d905fd8c7', 'request_id': 'req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:32.232804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] The token cache is being invalidate because project 8c4df9c4f14e4e44b0b66e2d905fd8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:32.233495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:32.235741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:32.241956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 972/1928] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:31 2022] DELETE /identity/v3/projects/8c4df9c4f14e4e44b0b66e2d905fd8c7 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:32.241956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.241956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.250101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:32.262216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.263111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.263744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.263744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.264327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.266192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3679c99-b98d-4286-938e-b94f38f4dffa admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.279912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 957/1929] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:32.281254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.281550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.281795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.303728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:32.307145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.314118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 958/1930] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:32.316304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.318081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.331027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d6e103-bfac-43f0-bd39-6825c6e0abed None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.345954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.347635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.347918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.348191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.348911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.350587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f15e3d-3323-4999-a130-e778c5e24311 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.352994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada00fb-bc0b-4c6d-920a-b846ac56f94f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.366458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 959/1931] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:32.366458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.366458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.366458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.368323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.369505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.370061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.370611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.371942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.375254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.398496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:32.399188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 973/1932] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:32.400060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.400445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.411253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 960/1933] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:32.412358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.417670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533d7ea-acb5-4ce8-8452-57b3b41b17a7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.424619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf57f-dcce-4739-8650-1d8cbceced49 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.425138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.426323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.426914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.427664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] PATH_INFO: `/v3/projects/e4506a17200e43f38a49576c6cc3c8aa/users/964b88e3f1174b72ae6d9ae52287f22f/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.430495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.431297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.431681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.432043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.432998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.435554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.449605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4506a17200e43f38a49576c6cc3c8aa, user_id=964b88e3f1174b72ae6d9ae52287f22f, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.454669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 961/1934] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:32.454669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.454925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.471219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b14d1b-cda5-4485-b4f7-7110442decb6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.472119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 974/1935] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:32 2022] PUT /identity/v3/projects/e4506a17200e43f38a49576c6cc3c8aa/users/964b88e3f1174b72ae6d9ae52287f22f/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:32.472697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.550200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.550953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.551341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.551689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.552441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:32.553012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.554645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818fee6d-3545-4861-8c07-b668430437cd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.575540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 962/1936] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:32.575845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9255b4d-90c1-49e8-8cf0-33bd795117a2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.576010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45615648-8ef6-438f-b933-f0445197b406 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.576010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45615648-8ef6-438f-b933-f0445197b406 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.576010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45615648-8ef6-438f-b933-f0445197b406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.582355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.583109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.583420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.583909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] PATH_INFO: `/v3/projects/458cd61897994af382f1f80919acb373/users/ad02d279ba7347b4ab30874c954c00bd/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.603863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] RBAC: Authorizing `identity:create_grant(project_id=458cd61897994af382f1f80919acb373, user_id=ad02d279ba7347b4ab30874c954c00bd, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.606568 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.607865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-45615648-8ef6-438f-b933-f0445197b406 None None] MFA Rules not processed for user `964b88e3f1174b72ae6d9ae52287f22f`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:32.609172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-45615648-8ef6-438f-b933-f0445197b406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.631203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 975/1937] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:32 2022] PUT /identity/v3/projects/458cd61897994af382f1f80919acb373/users/ad02d279ba7347b4ab30874c954c00bd/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:32.633325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.707720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 963/1938] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:32.709181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.709472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.709904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.732584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:32.746351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191383c1-04ca-4b83-8b3e-17693587613a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.757208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.758423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.759105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.759734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.761597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.763431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] MFA Rules not processed for user `ad02d279ba7347b4ab30874c954c00bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:32.765635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:32.766112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.779004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:32.821229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 976/1939] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/projects => generated 371 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:32.822579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.867166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 964/1940] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:32.868625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.885970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073baee-4255-487b-921a-ee19a91f0ffa None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.925231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0fe7d-9747-4bc5-9d1a-9e61aff2dccf admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:32.932855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:32.933789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:32.934243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:32.934690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:32.937542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:32.939788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:32.993809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 977/1941] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/projects => generated 363 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:32.995860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:32.998857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:33.004834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.006017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.006595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.007221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.008519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.011900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.048054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 965/1942] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:32 2022] POST /identity/v3/users => generated 286 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:33.049739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.068686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b1814-9ee0-439f-a4d6-c6aeb20f93d4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.098964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f43abe-38a4-4f2a-b649-eb9b5c5852ea admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.103516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.104143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.104427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.104707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.164638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.165292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.165579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.165846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.166503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.168576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.201264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 966/1943] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:33 2022] POST /identity/v3/users => generated 288 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:33.201264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.214311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.216180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.223528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7d5d-8f4b-4be3-87e7-e9c6f3ed51ca admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.231903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.233016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.233642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.234464 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.235994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.239476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.240008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 978/1944] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:33.242323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.259259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 967/1945] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:33.263990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.275246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f95e28e-cbd4-4b12-aa6a-e78474e88899 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.278748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e81f0b-6c73-4df2-9e55-36b62b069953 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.285104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.285756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.286515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.286793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.287046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.287450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.288451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.288813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.289282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.290187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.291349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.292606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.307819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 968/1946] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:33.311968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.318998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 979/1947] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:33.325617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.327570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b2c99-31e7-4c6a-851d-d7fa8d32c4ad admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.335753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.336584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.336864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.337364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] PATH_INFO: `/v3/projects/830ccc10c7284fc2942b49cd091e942a/users/bd8f3afb4bf040c58feb266b45c0f503/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.343208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:33.349278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59c8f98-f1d0-465a-8226-03c15c48afd6 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.362501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.366396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.366866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.368149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] PATH_INFO: `/v3/projects/277e5492203843f7a947251f7796a209/users/dd5459fd750143be9ead7906597691db/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.371585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] RBAC: Authorizing `identity:create_grant(project_id=830ccc10c7284fc2942b49cd091e942a, user_id=bd8f3afb4bf040c58feb266b45c0f503, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.375174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b7a3a-9724-40a3-a7a9-57a5ece7a511 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.394219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 969/1948] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:33 2022] PUT /identity/v3/projects/830ccc10c7284fc2942b49cd091e942a/users/bd8f3afb4bf040c58feb266b45c0f503/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:36:33.398757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:33.400682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.401154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.401433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.421191 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277e5492203843f7a947251f7796a209, user_id=dd5459fd750143be9ead7906597691db, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.424472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.425719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:33.427008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:33.452011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 980/1949] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:33 2022] PUT /identity/v3/projects/277e5492203843f7a947251f7796a209/users/dd5459fd750143be9ead7906597691db/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:33.453270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.543780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 970/1950] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:33.552836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.554215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:33.600520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694f4dc7-ac83-435a-8423-fafcd9c24a72 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.615797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.620904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.621516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.623297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.624784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.626968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.654276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 981/1951] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:33.659434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:33.661018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529257a1-474f-43ae-b5c0-e7819bd1a2bc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.673871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf297f3-ba52-481f-8e5a-18ecf2961817 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:33.691810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.692712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.693000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.693399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] PATH_INFO: `/v3/projects/277e5492203843f7a947251f7796a209/users/dd5459fd750143be9ead7906597691db/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.696442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277e5492203843f7a947251f7796a209, user_id=dd5459fd750143be9ead7906597691db, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.700401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe6d6a7-6259-44ab-bae3-1e7cad11cde3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.718826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 982/1952] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:33 2022] PUT /identity/v3/projects/277e5492203843f7a947251f7796a209/users/dd5459fd750143be9ead7906597691db/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:33.729497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.730202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.730503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.779727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:33.780702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:33.782206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:33.782206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:33.804281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] MFA Rules not processed for user `dd5459fd750143be9ead7906597691db`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:33.806851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:33.809601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:33.811218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:33.824810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 971/1953] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:36:33.937032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 983/1954] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:33.946070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:34.002708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:34.018678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b21aa51-3650-4e89-bdef-70aa2f0e3316 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:34.117717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:34.118798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:34.119358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:34.119877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:34.137418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:34.141650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85936dc8-ca83-45ae-bfd8-c2397bf5913f service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:34.164454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 972/1955] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:34.263713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:34.275683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2ee3b-f94c-497a-8a32-78746b2c60da None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:34.285864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:34.286658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:34.287283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:34.287506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:34.358895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:34.366081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:34.367985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:34.382469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 984/1956] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:35.135348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cef0e-1427-4154-a87c-8286f2250ae2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:35.136178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cef0e-1427-4154-a87c-8286f2250ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:35.136636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10cef0e-1427-4154-a87c-8286f2250ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:35.186995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c10cef0e-1427-4154-a87c-8286f2250ae2 None None] MFA Rules not processed for user `bd8f3afb4bf040c58feb266b45c0f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:35.189427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c10cef0e-1427-4154-a87c-8286f2250ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:35.334730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 973/1957] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:35.347954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:35.358426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a149a-891a-4bbe-a927-b003ffdb2146 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:35.376552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:35.376552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:35.376552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:35.376552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:35.402381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:35.408160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:35.476255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 985/1958] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:37.420085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46a13-4cc0-4e80-b44b-4ece8e8b9c61 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:37.421055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46a13-4cc0-4e80-b44b-4ece8e8b9c61 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:37.421055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b46a13-4cc0-4e80-b44b-4ece8e8b9c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:37.447066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d1b46a13-4cc0-4e80-b44b-4ece8e8b9c61 None None] MFA Rules not processed for user `bd8f3afb4bf040c58feb266b45c0f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:37.449288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d1b46a13-4cc0-4e80-b44b-4ece8e8b9c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:37.462823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 974/1959] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:37 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:37.852755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:37.964528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8683ec-3127-44ac-8dc4-a0375333cdb8 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:37.980585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:37.981776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:37.982493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:37.982758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] PATH_INFO: `/v3/users/ad02d279ba7347b4ab30874c954c00bd` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:37.984560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad02d279ba7347b4ab30874c954c00bd)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:37.986799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:38.070069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad02d279ba7347b4ab30874c954c00bd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.096618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad02d279ba7347b4ab30874c954c00bd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.098248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad02d279ba7347b4ab30874c954c00bd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.098950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad02d279ba7347b4ab30874c954c00bd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.110389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad02d279ba7347b4ab30874c954c00bd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.170722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 986/1960] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:37 2022] DELETE /identity/v3/users/ad02d279ba7347b4ab30874c954c00bd => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:38.176040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:38.176419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:38.176763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:38.222196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:38.224656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.348758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 975/1961] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:38.361678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:38.365337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.376838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:38.387592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.424068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f0bb-9ea6-47cc-839e-6ca72f0eb4cc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:38.433908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:38.435351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:38.436140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:38.436805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] PATH_INFO: `/v3/users/964b88e3f1174b72ae6d9ae52287f22f` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:38.439251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=964b88e3f1174b72ae6d9ae52287f22f)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:38.446949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:38.483976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:38.531121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '964b88e3f1174b72ae6d9ae52287f22f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.542402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '964b88e3f1174b72ae6d9ae52287f22f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.554967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca5fa5-7211-4339-9216-2cc2037d8818 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:38.558796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '964b88e3f1174b72ae6d9ae52287f22f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.559641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '964b88e3f1174b72ae6d9ae52287f22f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.560846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8f7cc767-524e-477a-88a3-52edeea6bcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '964b88e3f1174b72ae6d9ae52287f22f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:38.573010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:38.574335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:38.575541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:38.576645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:38.580096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.622771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 976/1962] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:38 2022] DELETE /identity/v3/users/964b88e3f1174b72ae6d9ae52287f22f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:38.629811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:38.631639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:38.634366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:38.634859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:38.635548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:38.647622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 987/1963] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:38.693009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:38.697222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.802127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 977/1964] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:38.814141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:38.816685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:38.838912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:38.846909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9235ff9e-c924-4de4-869a-fd53aae5d416 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:38.979058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:38.980167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:38.980623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:38.981060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.015784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:39.018585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee616e-a620-47c0-aa83-ae356095bae5 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:39.039040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 988/1965] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:39.189545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:39.196813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:39.280572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8e891-9aa5-44aa-b3a7-85f9e942200e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:39.287792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:39.288551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.288935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.289273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] PATH_INFO: `/v3/projects/458cd61897994af382f1f80919acb373` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.290596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=458cd61897994af382f1f80919acb373)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:39.292997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:39.442164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '458cd61897994af382f1f80919acb373', 'request_id': 'req-82141a59-5817-4f05-8f47-fe1272ec75a6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.457298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '458cd61897994af382f1f80919acb373', 'request_id': 'req-82141a59-5817-4f05-8f47-fe1272ec75a6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.481718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '458cd61897994af382f1f80919acb373', 'request_id': 'req-82141a59-5817-4f05-8f47-fe1272ec75a6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.527178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] The token cache is being invalidate because project 458cd61897994af382f1f80919acb373 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:39.527744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.528931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.533310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 978/1966] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:39 2022] DELETE /identity/v3/projects/458cd61897994af382f1f80919acb373 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:39.541732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d5b7-543d-45bb-b7f8-c2090c71c0db None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.542185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d5b7-543d-45bb-b7f8-c2090c71c0db None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.542546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d5b7-543d-45bb-b7f8-c2090c71c0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.591450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-8883d5b7-543d-45bb-b7f8-c2090c71c0db None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:39.592732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8883d5b7-543d-45bb-b7f8-c2090c71c0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:39.669920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:39.671708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:39.709122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 989/1967] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:39.712990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.713173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.714114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.734521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82141a59-5817-4f05-8f47-fe1272ec75a6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:39.744996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:39.746009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.746483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.747166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] PATH_INFO: `/v3/projects/e4506a17200e43f38a49576c6cc3c8aa` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.750412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4506a17200e43f38a49576c6cc3c8aa)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:39.764100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:39.767025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:39.772718 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:39.795514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 990/1968] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:39.796925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:39.813373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:39.816244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55080b24-c934-4cb0-a050-313e7a0ef91a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:39.822694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:39.823600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.823686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.823966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.824716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:39.825641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:39.827590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:39.854082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 991/1969] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:39.863226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:39.875163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4506a17200e43f38a49576c6cc3c8aa', 'request_id': 'req-afd7f628-a871-4cd4-8e10-0d83772f0d09'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.876961 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c88dd-9652-4c5d-be13-2655f1f026d4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:39.885298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4506a17200e43f38a49576c6cc3c8aa', 'request_id': 'req-afd7f628-a871-4cd4-8e10-0d83772f0d09'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.899901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4506a17200e43f38a49576c6cc3c8aa', 'request_id': 'req-afd7f628-a871-4cd4-8e10-0d83772f0d09'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.934068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] The token cache is being invalidate because project e4506a17200e43f38a49576c6cc3c8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:39.935237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.937095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:39.943480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 979/1970] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:39 2022] DELETE /identity/v3/projects/e4506a17200e43f38a49576c6cc3c8aa => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:39.947000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:39.949551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:39.962610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:39.963394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:39.963739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:39.964078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:39.964836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:39.967331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:39.982163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7f628-a871-4cd4-8e10-0d83772f0d09 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:39.997576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:39.999027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.003194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.004003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] PATH_INFO: `/v3/users/dd5459fd750143be9ead7906597691db` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.010140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd5459fd750143be9ead7906597691db)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.016582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 992/1971] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:39 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:40.017184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.018654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.020972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:40.055406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:40.096673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd5459fd750143be9ead7906597691db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.108267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd5459fd750143be9ead7906597691db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.123526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd5459fd750143be9ead7906597691db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.124625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd5459fd750143be9ead7906597691db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.125939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd5459fd750143be9ead7906597691db'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.149204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc4971-6c3f-4440-8373-1bb1aa17741b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.159513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.159513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.159513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.159968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.163131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.164994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.165122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 980/1972] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:36:39 2022] DELETE /identity/v3/users/dd5459fd750143be9ead7906597691db => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:40.213455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 993/1973] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:40 2022] POST /identity/v3/users => generated 289 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:40.214402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.217343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:40.357233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e94acb-dc37-47f4-b69a-ba00c1768503 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.363183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.364988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.364988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.364988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.365538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.367635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.380672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 981/1974] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:40.387051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.411450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd73e93-d283-4c4a-b6de-5cb4c2b0d441 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.417208 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.418639 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.418990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.419399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.420254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.422212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9f731a-fdd6-4ea1-ad60-cd2e629824b4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.440569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 994/1975] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:40.447894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.466861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2502a244-fcf8-49b1-bbfc-549c8cd46ed4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.496141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.497144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.497760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.498173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] PATH_INFO: `/v3/projects/f002a15e04f34f0da3cb28d425d09329/users/6748b2b169a242ed88f9559cd1be6d73/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.520516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:40.541108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f002a15e04f34f0da3cb28d425d09329, user_id=6748b2b169a242ed88f9559cd1be6d73, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.552680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.576337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 982/1976] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:40 2022] PUT /identity/v3/projects/f002a15e04f34f0da3cb28d425d09329/users/6748b2b169a242ed88f9559cd1be6d73/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:40.591213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.591782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.592320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.608087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.611919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:40.648716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] MFA Rules not processed for user `6748b2b169a242ed88f9559cd1be6d73`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:40.650308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:40.735032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf461-a711-4532-b3e1-44cfcad7ff89 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.740474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.741143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.741420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.741689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] PATH_INFO: `/v3/projects/277e5492203843f7a947251f7796a209` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.742917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277e5492203843f7a947251f7796a209)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.744976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.750014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 995/1977] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:40.754551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.769991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ed616-c128-45a2-98d1-f74449b88f22 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.775778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:40.776914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.776914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.777190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.778315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:40.780461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:40.828379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277e5492203843f7a947251f7796a209', 'request_id': 'req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.840173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 996/1978] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:40 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:36:40.844968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277e5492203843f7a947251f7796a209', 'request_id': 'req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.845465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:40.873445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277e5492203843f7a947251f7796a209', 'request_id': 'req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.889087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] The token cache is being invalidate because project 277e5492203843f7a947251f7796a209 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:36:40.889827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.890749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6a83b5d-51e5-43b5-83bc-81a0dfcd16a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:36:40.892928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 983/1979] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:36:40 2022] DELETE /identity/v3/projects/277e5492203843f7a947251f7796a209 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:40.897965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408be5e-b701-4e57-8436-68f929387f51 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:40.898232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408be5e-b701-4e57-8436-68f929387f51 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:40.898632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a408be5e-b701-4e57-8436-68f929387f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:40.923912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a408be5e-b701-4e57-8436-68f929387f51 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:40.925212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a408be5e-b701-4e57-8436-68f929387f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:40.966734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:40.974644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bece670-462a-47c5-961c-caf6f32d5dcd admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:40.999622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.000660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.001041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.001529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.002741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.007135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.020114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 984/1980] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:41.034052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.034478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.034859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.056728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 997/1981] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:40 2022] POST /identity/v3/users => generated 288 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:41.060192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:41.062787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:41.063873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.070134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:41.091889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 985/1982] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:36:41.099630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.110163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412c3d5-add2-491d-9cd1-83fd8f40fefb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.119387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b56de6-17e1-4dbc-9170-d7785a54594d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.124239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.125608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.126109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.126706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.128438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.129694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.130801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.131387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.131882 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.133138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:36:41.133359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.133696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.136849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.148874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:41.164604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 986/1983] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:36:41.165899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.170061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 998/1984] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:36:41.171995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.182991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67bdab-1af7-4563-aaa7-e9d227371234 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.193581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083e65fc-bc04-40f5-8e62-a0110fb21ebe admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.194354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.195629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.196237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.196810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.198054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.199177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.199826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.200109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.200384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.201136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.202353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.203222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.224890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 999/1985] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:41.227698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.239979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c44e0-b6bf-40a1-be49-ef69b526ea21 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.245151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 987/1986] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/projects => generated 377 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:41.245365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.246080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.246444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.246723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] PATH_INFO: `/v3/projects/9a180077e07e4011a24fd699ec9d47da/users/ea58d33b958a43898f6861f8c2bf3932/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.251653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.271824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a180077e07e4011a24fd699ec9d47da, user_id=ea58d33b958a43898f6861f8c2bf3932, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.277081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd24665-2f7d-485c-ac9e-6df9c17df2a5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.301898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1000/1987] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:41 2022] PUT /identity/v3/projects/9a180077e07e4011a24fd699ec9d47da/users/ea58d33b958a43898f6861f8c2bf3932/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:41.320319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.320805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.321274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.363598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a432b2-6f0a-4830-b689-b82ea1041b25 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.414240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] MFA Rules not processed for user `ea58d33b958a43898f6861f8c2bf3932`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:41.415948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:41.477759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.479832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.479832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.479832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.480316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.482842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.516170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 988/1988] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/users => generated 289 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:41.517349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.528820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85bc11-ed43-4e57-bfaf-218cfcd38b09 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.534992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.535698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.535962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.536226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.536917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.538630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.542023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1001/1989] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:41.546927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.556069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 989/1990] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:41.564736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.571841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b587a10-8781-48af-bf31-89f2aa8e05f7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.576724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.577361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.577645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.577995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.578659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.580708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.581822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308444a2-86ac-417f-8a91-70d83fee2e8c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.588104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.589328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.589912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.590159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.591003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.593363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.606523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 990/1991] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:41.614117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.625612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1002/1992] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:41.629830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.630285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc9fbe-ded6-414e-9462-cd7e47fdad94 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.690162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:41.705385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.706079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.706695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.706850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] PATH_INFO: `/v3/projects/e4dd80d3ffe84acc9e8c91f27fd6537d/users/b4e88e984e9a48b6ae9db39fcd422298/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.725884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4dd80d3ffe84acc9e8c91f27fd6537d, user_id=b4e88e984e9a48b6ae9db39fcd422298, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.728116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584220b7-44aa-4661-8010-e94cf8aef345 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.728759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf39bd-9e20-453d-94a6-a3c4bee142ed admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.734632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.735226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.735508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.735777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.736437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.738467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.746618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 991/1993] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:41 2022] PUT /identity/v3/projects/e4dd80d3ffe84acc9e8c91f27fd6537d/users/b4e88e984e9a48b6ae9db39fcd422298/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:41.760451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.760990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.761500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.774866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:41.784721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1003/1994] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/users => generated 294 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:41.788637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.801336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e7bad-6ade-42ec-bb12-5824bb38f819 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.809871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] MFA Rules not processed for user `b4e88e984e9a48b6ae9db39fcd422298`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:41.811205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:41.878436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.879032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.879432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.879885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.881061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.884215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.902352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 992/1995] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:41.907151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1004/1996] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:36:41.909940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.911110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:41.931374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d0f462-7b42-4427-97c3-e9dba6da652a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.934853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae1493e-0d0e-4da5-b318-d393edb998f5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:41.939799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.940869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.941347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.941755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] PATH_INFO: `/v3/projects/980dab7a80f443869adc8f7083cd127f/users/bbe0e3dacda14c699f3aac4bc594704d/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.942576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:41.944020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:41.944650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:41.945145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:41.946338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.949747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:41.970950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980dab7a80f443869adc8f7083cd127f, user_id=bbe0e3dacda14c699f3aac4bc594704d, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:41.976308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.012914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1005/1997] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:41 2022] PUT /identity/v3/projects/980dab7a80f443869adc8f7083cd127f/users/bbe0e3dacda14c699f3aac4bc594704d/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:42.018214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.051494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 993/1998] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:36:41 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:42.053981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.113715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5d1802-bf9c-4684-98e7-cc46df0ba4f8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.165201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07c0f6f-21d4-4ab5-bb67-7586903c62cd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.172609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.173695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.174196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.174701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.175945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.179891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.229013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 994/1999] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:36:42 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:36:42.242597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.244517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.244517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.244870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.246086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.251818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.255109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.270472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1006/2000] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:42.280624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.294254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f2b42a-4d2a-466c-b2bf-4fe4912a30c6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.304192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.305359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.305903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.306437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062ccf6c-a793-424c-85fd-1f2001313fd8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.306594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.307796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.311121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.316514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.317704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.318207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.318708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] PATH_INFO: `/v3/projects/980dab7a80f443869adc8f7083cd127f/users/bbe0e3dacda14c699f3aac4bc594704d/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.324997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980dab7a80f443869adc8f7083cd127f, user_id=bbe0e3dacda14c699f3aac4bc594704d, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.334910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 995/2001] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:42.335180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dd933-f58a-41fc-b919-86a3d330e8c4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.337853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.346030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:42.353081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1007/2002] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:42 2022] PUT /identity/v3/projects/980dab7a80f443869adc8f7083cd127f/users/bbe0e3dacda14c699f3aac4bc594704d/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:42.366758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.367747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.368946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.378050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69a54f-22de-404a-85ea-67bb18fbc2d6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.405143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] MFA Rules not processed for user `bbe0e3dacda14c699f3aac4bc594704d`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:42.406659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:42.418874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:42.491532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1008/2003] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:42.493021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.493780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.494509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.494509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] PATH_INFO: `/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b/users/347206a5850048d7923321da87c4fad7/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.504333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.506714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:42.513728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3eaf2d03f564b7eadf5cdac15e2043b, user_id=347206a5850048d7923321da87c4fad7, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.516734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.535009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 996/2004] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:42 2022] PUT /identity/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b/users/347206a5850048d7923321da87c4fad7/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:42.540985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.617042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420e0f68-2179-4e9a-af66-64af45ec36c1 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.666846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd39a9-acb1-45e0-b001-b382a39caa14 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.674723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.675540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.675911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.676241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.677121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.678939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.691308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 997/2005] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:36:42.697149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.710096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b030a3-e6cc-438d-b39e-71eea9de728e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:42.715515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.716187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.716619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.716893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] PATH_INFO: `/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b/users/347206a5850048d7923321da87c4fad7/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.719491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3eaf2d03f564b7eadf5cdac15e2043b, user_id=347206a5850048d7923321da87c4fad7, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.722010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e325c785-1439-424f-b958-232661a4d534 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.726235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:42.726990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.728150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.728634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.732638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:42.736451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 998/2006] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:36:42 2022] PUT /identity/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b/users/347206a5850048d7923321da87c4fad7/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:36:42.755505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:42.756080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:42.756851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:42.801254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] MFA Rules not processed for user `347206a5850048d7923321da87c4fad7`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:36:42.804578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:42.839107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:42.842327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:42.844083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:42.856897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1009/2007] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:42.956633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 999/2008] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:36:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:36:42.971881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:42.974064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:36:43.002280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23da944-21f0-4dfb-8de2-7f3c3295cff7 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:43.104344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:43.105055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:43.105451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:43.105850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:43.213310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:43.214949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:43.227485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1010/2009] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:45.099210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:45.116708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5322ca2f-de91-48dd-8590-7eb4d39e6bf2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:45.126545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:45.126545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:45.126545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:45.126545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:45.144711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:36:45.203942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:45.205389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:45.216787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1000/2010] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:46.211763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:46.221869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6901ec8-ac7d-4c7e-ab2b-5998b18fedf5 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:46.234924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:46.237876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:46.243633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:46.244180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:46.432407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:46.435134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:46.506545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1011/2011] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:46 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:51.693845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:36:51.706218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c3de2a-9896-437e-ae9b-e3f140ba58c8 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:36:51.710984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:36:51.711679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:36:51.711988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:36:51.712307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:36:51.724949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:36:51.727767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c59b2-9130-4cf8-b089-c3e93687ca19 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:36:51.743574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1001/2012] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:36:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:36:56.461500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=80 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:00.513225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:00.586120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4572519-9eca-48d6-a268-266cbc0a7fb4 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:00.591394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:00.592062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:00.592348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:00.592624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] PATH_INFO: `/v3/users/6748b2b169a242ed88f9559cd1be6d73` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:00.593845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6748b2b169a242ed88f9559cd1be6d73)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:00.595991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:00.650844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6748b2b169a242ed88f9559cd1be6d73'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:00.658319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6748b2b169a242ed88f9559cd1be6d73'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:00.659063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6748b2b169a242ed88f9559cd1be6d73'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:00.659709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6748b2b169a242ed88f9559cd1be6d73'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:00.669442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6748b2b169a242ed88f9559cd1be6d73'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:00.689631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1012/2013] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:00 2022] DELETE /identity/v3/users/6748b2b169a242ed88f9559cd1be6d73 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:00.696634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:00.697353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:00.697753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:00.724787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:00.726088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:00.798940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1002/2014] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:00.825021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:00.826632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:00.913809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66789744-edf2-415b-b5d5-cced9332c7f8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:00.918160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:00.918800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:00.919103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:00.919369 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:00.930915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:00.932734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:00.947474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1013/2015] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:37:01.371319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:01.373507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:01.397172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fce230-0532-4a1a-8621-04c894a6ddbe None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:01.402986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:01.403706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:01.403972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:01.404402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] PATH_INFO: `/v3/projects/f002a15e04f34f0da3cb28d425d09329` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:01.406221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f002a15e04f34f0da3cb28d425d09329)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:01.408599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:01.469945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f002a15e04f34f0da3cb28d425d09329', 'request_id': 'req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.476948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f002a15e04f34f0da3cb28d425d09329', 'request_id': 'req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.490320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f002a15e04f34f0da3cb28d425d09329', 'request_id': 'req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.510660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] The token cache is being invalidate because project f002a15e04f34f0da3cb28d425d09329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:01.511130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.512043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.514048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1003/2016] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:01 2022] DELETE /identity/v3/projects/f002a15e04f34f0da3cb28d425d09329 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:01.519619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:01.532127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a272732-4308-4e54-b5c6-b8e55549c09b service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:01.555248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:01.616789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:01.617731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:01.618101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:01.618485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] PATH_INFO: `/v3/users/ea58d33b958a43898f6861f8c2bf3932` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:01.619958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea58d33b958a43898f6861f8c2bf3932)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:01.623343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:01.697605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea58d33b958a43898f6861f8c2bf3932'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.710705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea58d33b958a43898f6861f8c2bf3932'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.711777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea58d33b958a43898f6861f8c2bf3932'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.712636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea58d33b958a43898f6861f8c2bf3932'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.725808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea58d33b958a43898f6861f8c2bf3932'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:01.752358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1014/2017] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:01 2022] DELETE /identity/v3/users/ea58d33b958a43898f6861f8c2bf3932 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:02.154118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:02.156720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:02.179164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48f01e-42d0-4c8b-9c42-5e5b0569de42 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:02.201514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:02.262969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:02.263723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:02.263999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:02.264278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] PATH_INFO: `/v3/projects/9a180077e07e4011a24fd699ec9d47da` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:02.265521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a180077e07e4011a24fd699ec9d47da)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:02.267632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:02.332879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a180077e07e4011a24fd699ec9d47da', 'request_id': 'req-068b4c3f-d39c-432b-b418-55949fb1e803'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.339540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a180077e07e4011a24fd699ec9d47da', 'request_id': 'req-068b4c3f-d39c-432b-b418-55949fb1e803'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.348115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a180077e07e4011a24fd699ec9d47da', 'request_id': 'req-068b4c3f-d39c-432b-b418-55949fb1e803'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.361655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] The token cache is being invalidate because project 9a180077e07e4011a24fd699ec9d47da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:02.361951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.362846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.365006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1004/2018] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:02 2022] DELETE /identity/v3/projects/9a180077e07e4011a24fd699ec9d47da => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:02.371375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:02.373865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:02.391143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf12af-a6a2-496d-972f-3c3ae0315863 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:02.463535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:02.464496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:02.464953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:02.465416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] PATH_INFO: `/v3/users/bbe0e3dacda14c699f3aac4bc594704d` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:02.467392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe0e3dacda14c699f3aac4bc594704d)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:02.470498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:02.531810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:02.536428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe0e3dacda14c699f3aac4bc594704d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.548089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe0e3dacda14c699f3aac4bc594704d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.548860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe0e3dacda14c699f3aac4bc594704d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.549771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe0e3dacda14c699f3aac4bc594704d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.562898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0eeebdce-1a80-41cb-b303-70f15ad9f16e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe0e3dacda14c699f3aac4bc594704d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:02.591603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1015/2019] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:02 2022] DELETE /identity/v3/users/bbe0e3dacda14c699f3aac4bc594704d => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:03.079910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.081911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:03.101196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b4c3f-d39c-432b-b418-55949fb1e803 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.166156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.166965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.167279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.167554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] PATH_INFO: `/v3/projects/980dab7a80f443869adc8f7083cd127f` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.168817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980dab7a80f443869adc8f7083cd127f)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.172314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.244340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980dab7a80f443869adc8f7083cd127f', 'request_id': 'req-9d811736-f7fb-4051-a57a-43d1983583b3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:03.247165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:03.255550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980dab7a80f443869adc8f7083cd127f', 'request_id': 'req-9d811736-f7fb-4051-a57a-43d1983583b3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:03.268956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980dab7a80f443869adc8f7083cd127f', 'request_id': 'req-9d811736-f7fb-4051-a57a-43d1983583b3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:03.288858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] The token cache is being invalidate because project 980dab7a80f443869adc8f7083cd127f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:03.289334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:03.290127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9d811736-f7fb-4051-a57a-43d1983583b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:03.296236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1005/2020] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:03 2022] DELETE /identity/v3/projects/980dab7a80f443869adc8f7083cd127f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:03.298811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.299398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.299943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.334787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:03.337268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:03.430726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1016/2021] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:03.440305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.440584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.440864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.462071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:03.463400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:03.472344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1006/2022] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:03.477364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.490400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb18250-1c54-443c-86c6-9196f567c3e8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.498275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.499406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.499885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.500400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.501604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:03.502160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.505339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.528065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1017/2023] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:03.536833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.547234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf26f5-f739-47ed-b4be-654d461443cc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.553242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.553886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.554161 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.554424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.555066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.557073 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.589696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1007/2024] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:03 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:03.594532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.606413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd13b360-a340-4cde-84be-209d3850ca5a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.669613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.670239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.670523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.670791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.671462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.673475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.705595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1018/2025] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:03 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:03.709649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.722962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26223cd-4f89-49ad-bab6-8006e9160e9b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.728358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.728992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.729273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.729531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.730192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.731872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.748075 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1008/2026] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:03.749463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.760352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fdc4f0-7328-4569-a11d-91bf1ecf6d2b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.765431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.766090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.766372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.766640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.767330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.768946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a9eac-205d-49bc-bfd6-33341e41a02e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.788384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1019/2027] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:03.789555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.801893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121ba1-408a-4bb3-b1a1-c0e70acdf4c4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:03.806891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:03.807758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.808035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.808311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] PATH_INFO: `/v3/projects/70e5049c77884c4c85231ffdfbe90509/users/2749f7d863ba4903b32f9f44fbfff28d/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.825474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e5049c77884c4c85231ffdfbe90509, user_id=2749f7d863ba4903b32f9f44fbfff28d, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:03.828349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:03.845519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1009/2028] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:03 2022] PUT /identity/v3/projects/70e5049c77884c4c85231ffdfbe90509/users/2749f7d863ba4903b32f9f44fbfff28d/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:03.856373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a2b2-e6b2-4d4d-805b-e2df2598d61a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:03.856661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a2b2-e6b2-4d4d-805b-e2df2598d61a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:03.856934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4a2b2-e6b2-4d4d-805b-e2df2598d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:03.889994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-50a4a2b2-e6b2-4d4d-805b-e2df2598d61a None None] MFA Rules not processed for user `2749f7d863ba4903b32f9f44fbfff28d`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:03.891345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-50a4a2b2-e6b2-4d4d-805b-e2df2598d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:03.909877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:03.967331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1020/2029] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:03.982237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:03.984026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:04.044155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552841fc-882f-438a-8d9c-368aead77d40 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:04.047985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:04.048603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:04.048878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:04.049146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:04.058371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:04.059861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7b853-7e10-4799-bb3a-0608f766a10d service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:04.070671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1010/2030] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:37:13.964141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=20 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:14.065355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:21.378904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:21.379611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:21.379706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:21.381983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1021/2031] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:37:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:37:21.631844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:21.632830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:21.633104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:21.656202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] MFA Rules not processed for user `44983e1fc3fc4079a31d1e078fb475fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:21.657492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:21.701987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:21.704279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:21.766326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1011/2032] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:21.780177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:21.782134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:21.797835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602be2-b8b0-4397-b375-aa302d88e614 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:21.809686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:21.810778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:21.811288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:21.811769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:21.814223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bfd884-d061-43f1-857f-2656b5b64fc2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:21.820827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:21.821872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:21.822326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:21.822747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] PATH_INFO: `/v3/users/c93191a5e3c7476f926b56424ccec4ba` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:21.824549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c93191a5e3c7476f926b56424ccec4ba)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:21.827382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:21.829231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:21.832441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:21.852951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1012/2033] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:21.920125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c93191a5e3c7476f926b56424ccec4ba'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:21.929438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c93191a5e3c7476f926b56424ccec4ba'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:21.930189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c93191a5e3c7476f926b56424ccec4ba'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:21.930822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c93191a5e3c7476f926b56424ccec4ba'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:21.942253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-9ffd7200-a369-489a-82aa-2703d387d4e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c93191a5e3c7476f926b56424ccec4ba'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:21.969407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1022/2034] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:21 2022] DELETE /identity/v3/users/c93191a5e3c7476f926b56424ccec4ba => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:22.184217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.197710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a230498-171f-4ea9-a6d5-1897c3b4082c service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.287281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.287675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.287975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.288322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] PATH_INFO: `/v3/projects/5021d13535274fa2b44666c8d39080bf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.290117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5021d13535274fa2b44666c8d39080bf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.293900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.351969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5021d13535274fa2b44666c8d39080bf', 'request_id': 'req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:22.357471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5021d13535274fa2b44666c8d39080bf', 'request_id': 'req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:22.365653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5021d13535274fa2b44666c8d39080bf', 'request_id': 'req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:22.377889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] The token cache is being invalidate because project 5021d13535274fa2b44666c8d39080bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:22.378189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:22.378928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9be38c94-ffda-4ab8-8a2f-7e7730d921dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:22.380917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1013/2035] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:22 2022] DELETE /identity/v3/projects/5021d13535274fa2b44666c8d39080bf => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:22.388885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.389335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.389441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.412658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:22.413895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:22.490443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1023/2036] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:22.492671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.493683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.494106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.518341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:22.519887 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:22.528004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1014/2037] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:22.532500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.542462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d1a69-6e68-4e56-837b-30b7d717f77d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.548760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.549387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.549671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.549942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.550815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:22.551159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.555710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.571108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1024/2038] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:22.572942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.582632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4487e6-abe7-49aa-a3d0-7cfbceb91f59 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.587289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.587900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.588175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.588525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.589595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.591562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.596987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:22.627033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1015/2039] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:22 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:22.631349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.643118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba601f7-5098-4c76-81c8-41b52487a369 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.647686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:22.726319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.727441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.727904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.728295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.729034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.731621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.767717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1025/2040] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:22 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:22.769541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.786136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657be755-febd-4570-ab9a-117cfa5e1610 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.791932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.792576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.792925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.793233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.793889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.795568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.809409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1016/2041] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:22.813335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.826788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b613322b-30f4-468a-9346-ddd6ab67fcd7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.832087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.832907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.833186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.833486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.834192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.835929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31c095e-f28c-4697-9656-aed92d50ea34 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.851118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1026/2042] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:37:22.853539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:22.865802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef92f3e-d390-4ecb-b0de-2dba9612eb07 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:22.876788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:22.877562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.877840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.878156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] PATH_INFO: `/v3/projects/8a9e3b846df943e0838de5bc7c31b418/users/b68f03248ac84002899e496286e9a24f/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:22.902085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9e3b846df943e0838de5bc7c31b418, user_id=b68f03248ac84002899e496286e9a24f, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:22.905375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:22.963156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1017/2043] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:22 2022] PUT /identity/v3/projects/8a9e3b846df943e0838de5bc7c31b418/users/b68f03248ac84002899e496286e9a24f/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:22.969133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:22.970222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:22.970959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.070130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] MFA Rules not processed for user `b68f03248ac84002899e496286e9a24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:23.072009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:23.213035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1027/2044] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:23.224261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.301398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2f933-9f45-4c74-9a72-4c6c6b16e0fb admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.307536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.308456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.309003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.309553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.311726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.314668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.349141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1018/2045] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:23 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:23.350795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.365626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081953f-d6f1-4969-9cc5-d509fc2c3f9f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.432410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.433090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.433291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.433560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.434317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.436656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.461018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1028/2046] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:23 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:23.466557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.478142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e324c8-1f52-4f04-9e16-bd9aeecdb9bb admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.485592 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.486545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.486545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.486796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.487796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.489450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.501855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1019/2047] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:23.506839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.514654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:23.518887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447ecc0-4a10-4419-8679-e0e212579c96 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.527055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.527804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.528119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.528399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] PATH_INFO: `/v3/projects/0e5706b8531a4fbdb091069bec143a11/users/843f70e76d704de98a30497c2d1ed3e6/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.552461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5706b8531a4fbdb091069bec143a11, user_id=843f70e76d704de98a30497c2d1ed3e6, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.555559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.572978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1029/2048] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:23 2022] PUT /identity/v3/projects/0e5706b8531a4fbdb091069bec143a11/users/843f70e76d704de98a30497c2d1ed3e6/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:23.577284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.590631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc690eda-b1da-4c22-80cd-bee95b18e726 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.677240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.677240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.677240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.677240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.677606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.678866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc2138f-510d-43d4-a933-3a2d349cdba6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.688933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1020/2049] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:23 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:23.693196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.703705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfd647-cb74-4c2e-94cf-20b2a5d8de0b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.708233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.708881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.709159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.709430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] PATH_INFO: `/v3/projects/0e5706b8531a4fbdb091069bec143a11/users/843f70e76d704de98a30497c2d1ed3e6/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.711423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5706b8531a4fbdb091069bec143a11, user_id=843f70e76d704de98a30497c2d1ed3e6, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:23.714032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:23.734585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1030/2050] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:23 2022] PUT /identity/v3/projects/0e5706b8531a4fbdb091069bec143a11/users/843f70e76d704de98a30497c2d1ed3e6/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:23.748821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.749108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.749495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:23.781484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] MFA Rules not processed for user `843f70e76d704de98a30497c2d1ed3e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:23.782774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:23.794240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:23.869105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1021/2051] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:23 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:23.882603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:23.884392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:23.988764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fbfdf-b10c-441e-924e-27edeb50543a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:23.988764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:23.989271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:23.989578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:23.990144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:24.002506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:24.004074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:24.015970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1031/2052] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:23 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:24.126386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:24.138691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbbbbe2-e136-466b-b928-350724d2b5a5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:24.143398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:24.145222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:24.145446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:24.145699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:24.217163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:24.218671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef65ad-387b-423a-85e1-4e69449a4541 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:24.229451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1022/2053] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:24.708430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:24.710637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:24.797217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af20e907-eb29-486f-95bc-a1f56e67390f service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:24.802038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:24.802691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:24.802966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:24.803268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] PATH_INFO: `/v3/users/2749f7d863ba4903b32f9f44fbfff28d` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:24.805466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2749f7d863ba4903b32f9f44fbfff28d)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:24.807649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:24.863632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2749f7d863ba4903b32f9f44fbfff28d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:24.871179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2749f7d863ba4903b32f9f44fbfff28d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:24.871920 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2749f7d863ba4903b32f9f44fbfff28d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:24.872526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2749f7d863ba4903b32f9f44fbfff28d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:24.881893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2749f7d863ba4903b32f9f44fbfff28d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:24.890715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:24.905479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1032/2054] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:24 2022] DELETE /identity/v3/users/2749f7d863ba4903b32f9f44fbfff28d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:24.910514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:24.911071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:24.911229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:24.940792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:24.942664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:25.036313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1023/2055] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:25.049744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:25.051545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:25.126051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d64d0-d847-4ac4-bd93-71465faa8d36 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:25.130184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:25.130815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:25.131101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:25.131383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:25.141946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:25.143463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a7358-31f7-4e71-bd9e-2095c3d72fe6 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:25.154600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1033/2056] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:25.583170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:25.584815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:25.612901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68960cb2-9b45-4a39-a481-fa3220df8592 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:25.621281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:25.622245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:25.622462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:25.622981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] PATH_INFO: `/v3/projects/70e5049c77884c4c85231ffdfbe90509` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:25.625252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e5049c77884c4c85231ffdfbe90509)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:25.629186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:25.731194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:25.748272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e5049c77884c4c85231ffdfbe90509', 'request_id': 'req-d6a6900b-58b8-473e-913f-27b648e5381c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:25.761287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e5049c77884c4c85231ffdfbe90509', 'request_id': 'req-d6a6900b-58b8-473e-913f-27b648e5381c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:25.775392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e5049c77884c4c85231ffdfbe90509', 'request_id': 'req-d6a6900b-58b8-473e-913f-27b648e5381c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:25.792170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] The token cache is being invalidate because project 70e5049c77884c4c85231ffdfbe90509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:25.792564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:25.794539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d6a6900b-58b8-473e-913f-27b648e5381c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:25.796249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1024/2057] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:25 2022] DELETE /identity/v3/projects/70e5049c77884c4c85231ffdfbe90509 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:25.803199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:25.824339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:25.824664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:25.860241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:25.861562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:25.970466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1034/2058] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:25.972732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:25.973315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:25.973693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.006329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:26.007962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:26.019417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1025/2059] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:26.024552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.045710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c005a6f-b197-4103-9906-6b4f7b890e75 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.060149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.060149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.060149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.060149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.060496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:26.060496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.062740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.087214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.095985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1035/2060] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:26.113590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23d333-c634-4cad-9ca2-efac5e6732ce None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.121950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.122698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.123134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.123570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.124421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.127837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.167115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1026/2061] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:26 2022] POST /identity/v3/projects => generated 399 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:26.171899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.183814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0eaaeb-7f82-401d-acb0-98a374d091a8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.256388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.257213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.257638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.257980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.258698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.261154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.287759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1036/2062] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:26 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:26.292654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.304023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37407579-95c3-4df7-8c13-a6017f4658a4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.309961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.310610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.310885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.311193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.314068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.314068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.326169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1027/2063] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:26.330777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.342924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38f78bf-b008-4b41-9c6d-ab89d41ba4e5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.349237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.349889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.350167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.350433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.351197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.352897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bd1a29-07f7-43b0-b134-2dc0051522ba admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.355902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:26.369096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1037/2064] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:26.373517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.387546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c543104-646a-46a0-9acb-055ac29a3949 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.394057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.395148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.395588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.396104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] PATH_INFO: `/v3/projects/abd21af5094241d68fe66d1a9a879ec3/users/ee51b102e38347d6a613bf15f9e63572/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.423585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd21af5094241d68fe66d1a9a879ec3, user_id=ee51b102e38347d6a613bf15f9e63572, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.429499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.447871 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1028/2065] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:26 2022] PUT /identity/v3/projects/abd21af5094241d68fe66d1a9a879ec3/users/ee51b102e38347d6a613bf15f9e63572/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:26.460841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.461242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.461515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.492930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] MFA Rules not processed for user `ee51b102e38347d6a613bf15f9e63572`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:26.494186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:26.568153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1038/2066] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:26 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:26.585107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.587264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:26.654233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4c33c-29b8-4cd3-b400-89259adbc40c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.657984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.659531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.660151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.660447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.670971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.672624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a234d6c2-91ac-4c7e-bfce-9d80d9c34d3a service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:26.685894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1029/2067] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:26 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:26.817051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:26.819424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:26.975807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6a7c60-9d73-4af0-a576-25cbd3335d03 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:26.983246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:26.984373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:26.984878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:26.985210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] PATH_INFO: `/v3/users/b4e88e984e9a48b6ae9db39fcd422298` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:26.986882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e88e984e9a48b6ae9db39fcd422298)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:26.989633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:27.102246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e88e984e9a48b6ae9db39fcd422298'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:27.114744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e88e984e9a48b6ae9db39fcd422298'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:27.115757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e88e984e9a48b6ae9db39fcd422298'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:27.116759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e88e984e9a48b6ae9db39fcd422298'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:27.129494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e88e984e9a48b6ae9db39fcd422298'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:27.159162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1039/2068] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:26 2022] DELETE /identity/v3/users/b4e88e984e9a48b6ae9db39fcd422298 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:27.166468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:27.167309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:27.167512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:27.214458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:27.216989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:27.404444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1030/2069] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:27.423591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:27.426131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:27.451626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:27.556416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b199d-864e-4b7a-8909-588c54d9d6a4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:27.561113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:27.562017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:27.562432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:27.562634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:27.581751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:27.584605 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:27.605518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1040/2070] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:37:28.179141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:28.180773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:28.194797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b69426d-8d9d-41c6-81e4-aaad02328682 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:28.200931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:28.201998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:28.202451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:28.202788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] PATH_INFO: `/v3/projects/e4dd80d3ffe84acc9e8c91f27fd6537d` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:28.204156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4dd80d3ffe84acc9e8c91f27fd6537d)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:28.206283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:28.216606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:28.283824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4dd80d3ffe84acc9e8c91f27fd6537d', 'request_id': 'req-bca9f48f-928e-429a-bc63-9e0027710935'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.292496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4dd80d3ffe84acc9e8c91f27fd6537d', 'request_id': 'req-bca9f48f-928e-429a-bc63-9e0027710935'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.306083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4dd80d3ffe84acc9e8c91f27fd6537d', 'request_id': 'req-bca9f48f-928e-429a-bc63-9e0027710935'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.332107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] The token cache is being invalidate because project e4dd80d3ffe84acc9e8c91f27fd6537d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:28.332956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.334484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.337993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1031/2071] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:28 2022] DELETE /identity/v3/projects/e4dd80d3ffe84acc9e8c91f27fd6537d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:28.345675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:28.374260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3332e-0ca4-48eb-b0b3-a859521e2f56 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:28.459814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:28.461121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:28.461949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:28.462170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] PATH_INFO: `/v3/users/347206a5850048d7923321da87c4fad7` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:28.464376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] RBAC: Authorizing `identity:delete_user(user_id=347206a5850048d7923321da87c4fad7)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:28.468166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:28.570026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347206a5850048d7923321da87c4fad7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.581222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347206a5850048d7923321da87c4fad7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.581982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347206a5850048d7923321da87c4fad7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.584051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347206a5850048d7923321da87c4fad7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.594222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-34a4abcb-c616-42fd-be89-45a3e5b1384d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347206a5850048d7923321da87c4fad7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:28.617151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1041/2072] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:28 2022] DELETE /identity/v3/users/347206a5850048d7923321da87c4fad7 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:28.996983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:28.998663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:29.022496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9f48f-928e-429a-bc63-9e0027710935 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.086673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.087361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.087638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.087909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] PATH_INFO: `/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.089056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3eaf2d03f564b7eadf5cdac15e2043b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.090978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.145679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3eaf2d03f564b7eadf5cdac15e2043b', 'request_id': 'req-227379a5-5b75-47f5-838a-0d542c9aaa07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:29.154510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3eaf2d03f564b7eadf5cdac15e2043b', 'request_id': 'req-227379a5-5b75-47f5-838a-0d542c9aaa07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:29.169072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3eaf2d03f564b7eadf5cdac15e2043b', 'request_id': 'req-227379a5-5b75-47f5-838a-0d542c9aaa07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:29.182280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] The token cache is being invalidate because project b3eaf2d03f564b7eadf5cdac15e2043b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:29.182941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:29.183521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-227379a5-5b75-47f5-838a-0d542c9aaa07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:29.185066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1032/2073] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:28 2022] DELETE /identity/v3/projects/b3eaf2d03f564b7eadf5cdac15e2043b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:29.192134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.192722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.193036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.215104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:29.216368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:29.244891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:29.291122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1042/2074] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:29.299575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.299946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.300256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.322063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:29.323398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:29.332016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1033/2075] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:29.336573 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.347497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff612dd-8428-46a0-a73e-690fbbf8300d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.352765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.353397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.353659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.353951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.354601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:29.354923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.356718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.370730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1043/2076] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:29.377659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.391404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac66282e-3f6d-42b5-abe4-b625eaf1a2fe None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.396743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.397376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.397648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.397903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.398548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.400556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.413316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:29.433508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1034/2077] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:29.439252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.450634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b8327-de33-4227-a8ac-18560edf7942 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.523019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.524100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.524580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.525083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.526312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.530178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.557786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1044/2078] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:29.562845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.575983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f1f33-ebd4-44c6-871d-ed45cc701339 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.583392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.584313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.584692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.585047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.585854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.587728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.600196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1035/2079] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:29.604831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.616414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64500c8f-b277-4c18-9819-7ed631c1a74a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.624914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.625987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.626494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.626988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.628291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.631516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeb5dac-9e5b-44a4-a1ae-31e22c01d816 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.648416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1045/2080] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:37:29.653532 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.664368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b9444-479a-42c8-b065-2179736ba021 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.669561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.670258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.670536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.670805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] PATH_INFO: `/v3/projects/febd124768ac461484673c8ea9a60d08/users/1e73bd5bea474fc9a19f4aa89d6c6ab5/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.687601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=febd124768ac461484673c8ea9a60d08, user_id=1e73bd5bea474fc9a19f4aa89d6c6ab5, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.690550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.706034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1036/2081] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:29 2022] PUT /identity/v3/projects/febd124768ac461484673c8ea9a60d08/users/1e73bd5bea474fc9a19f4aa89d6c6ab5/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:29.718968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.719301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.719558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.753386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] MFA Rules not processed for user `1e73bd5bea474fc9a19f4aa89d6c6ab5`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:29.754652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:29.826727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1046/2082] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:29.831757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.901174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245eb7a-81d3-42b0-884f-c968d86655a4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:29.907658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:29.908275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:29.908551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:29.908846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:29.909447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:29.911393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:29.949116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1037/2083] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:29.955544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:29.974773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf1186a-e3b6-4bd6-b797-e20f25cfe0f6 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.004869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:30.049114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.049716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.049989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.050255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.050890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.053179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.077772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1047/2084] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:29 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:30.082039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.097786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8945dd-c8a3-4098-85dc-01da176643b0 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.102735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.103385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.103661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.103927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.104575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.106181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.119898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1038/2085] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:30.124488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.134592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4ee0eb-5625-4174-ae95-4258e9c01ca5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.139951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.140710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.140986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.141278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] PATH_INFO: `/v3/projects/796a31507f6243edb9146fd9b43bcb4c/users/630a55fb672247b19a713c79e7d3e597/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.158679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a31507f6243edb9146fd9b43bcb4c, user_id=630a55fb672247b19a713c79e7d3e597, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.161338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.180646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1048/2086] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:30 2022] PUT /identity/v3/projects/796a31507f6243edb9146fd9b43bcb4c/users/630a55fb672247b19a713c79e7d3e597/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:30.184722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.195325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b439c-9f0c-41ca-ba8d-21dc750ba766 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.256439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.257115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.257397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.257660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.258314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.259994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d87151d-b9ff-430c-8eef-f612d485cdbd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.271044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1039/2087] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:37:30.276569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.288455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb391ec8-199e-4268-b1d1-7369a5e79f60 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.293515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.294167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.294440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.294707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] PATH_INFO: `/v3/projects/796a31507f6243edb9146fd9b43bcb4c/users/630a55fb672247b19a713c79e7d3e597/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.296658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=796a31507f6243edb9146fd9b43bcb4c, user_id=630a55fb672247b19a713c79e7d3e597, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.299329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.310689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1049/2088] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:30 2022] PUT /identity/v3/projects/796a31507f6243edb9146fd9b43bcb4c/users/630a55fb672247b19a713c79e7d3e597/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:30.321243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.321496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.321798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.356083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] MFA Rules not processed for user `630a55fb672247b19a713c79e7d3e597`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:30.357215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:30.372739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:30.435120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1040/2089] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:30.450232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.452665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:30.514578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de5546f-8df6-4946-9cbc-a62416cada2b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.518724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.519318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.519593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.519863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.529812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.531391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b786f2-5e25-4934-ab71-f9cb932d185c service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.542521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1050/2090] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:30.585516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.663613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caaee4-a4a6-4fda-b28f-e059cc28833c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.669263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.669947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.670250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.670632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] PATH_INFO: `/v3/users/1e73bd5bea474fc9a19f4aa89d6c6ab5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.671776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e73bd5bea474fc9a19f4aa89d6c6ab5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.673747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.731234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e73bd5bea474fc9a19f4aa89d6c6ab5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:30.739371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e73bd5bea474fc9a19f4aa89d6c6ab5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:30.748361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e73bd5bea474fc9a19f4aa89d6c6ab5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:30.748961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e73bd5bea474fc9a19f4aa89d6c6ab5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:30.749734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e73bd5bea474fc9a19f4aa89d6c6ab5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:30.769139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1041/2091] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:30 2022] DELETE /identity/v3/users/1e73bd5bea474fc9a19f4aa89d6c6ab5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:30.774116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.774523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.774916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.794844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:30.796129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:30.867163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1051/2092] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:30.880318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:30.881911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:30.923299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:30.952034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2582b-cbaf-45ad-b965-07f7f1d7cc99 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:30.956039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:30.956706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:30.956983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:30.957255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:30.967199 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:30.968762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:30.979264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1042/2093] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:31.298012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:31.299739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:31.319238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a4256-3bcb-4012-b278-aed6d29525d3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:31.325873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:31.326492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:31.326960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:31.327494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] PATH_INFO: `/v3/projects/febd124768ac461484673c8ea9a60d08` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:31.329632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=febd124768ac461484673c8ea9a60d08)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:31.333562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:31.355488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:31.397964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'febd124768ac461484673c8ea9a60d08', 'request_id': 'req-13005c3d-976a-4934-b386-4b3d8ceded96'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.405141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'febd124768ac461484673c8ea9a60d08', 'request_id': 'req-13005c3d-976a-4934-b386-4b3d8ceded96'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.414864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'febd124768ac461484673c8ea9a60d08', 'request_id': 'req-13005c3d-976a-4934-b386-4b3d8ceded96'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.427891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] The token cache is being invalidate because project febd124768ac461484673c8ea9a60d08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:31.428170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.428936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.430548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1052/2094] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:31 2022] DELETE /identity/v3/projects/febd124768ac461484673c8ea9a60d08 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:31.434927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:31.447447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7ce89-dba2-4799-b988-0cf396e36a07 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:31.514352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:31.514982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:31.515351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:31.515649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] PATH_INFO: `/v3/users/630a55fb672247b19a713c79e7d3e597` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:31.516984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630a55fb672247b19a713c79e7d3e597)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:31.519015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:31.571466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630a55fb672247b19a713c79e7d3e597'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.578760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630a55fb672247b19a713c79e7d3e597'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.587890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630a55fb672247b19a713c79e7d3e597'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.588632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630a55fb672247b19a713c79e7d3e597'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.589247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3fe66657-0f51-4f7b-b8c6-d55f9ed581e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630a55fb672247b19a713c79e7d3e597'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:31.609779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1043/2095] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:31 2022] DELETE /identity/v3/users/630a55fb672247b19a713c79e7d3e597 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:31.909557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:31.911894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:31.928143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13005c3d-976a-4934-b386-4b3d8ceded96 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.004640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.005435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.005721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.005998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] PATH_INFO: `/v3/projects/796a31507f6243edb9146fd9b43bcb4c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.007169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=796a31507f6243edb9146fd9b43bcb4c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.009113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.065848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '796a31507f6243edb9146fd9b43bcb4c', 'request_id': 'req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:32.072733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '796a31507f6243edb9146fd9b43bcb4c', 'request_id': 'req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:32.083574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '796a31507f6243edb9146fd9b43bcb4c', 'request_id': 'req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:32.098684 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] The token cache is being invalidate because project 796a31507f6243edb9146fd9b43bcb4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:32.098968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:32.099653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0b743a2c-ffa1-4c06-91a7-c3c7dbd631cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:32.101176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1053/2096] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:31 2022] DELETE /identity/v3/projects/796a31507f6243edb9146fd9b43bcb4c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:32.106824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.107101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.107432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.128389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:32.129693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:32.184633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:32.198618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1044/2097] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:32.206826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.207041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.207390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.230553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:32.231888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:32.242026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1054/2098] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:32.246844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.258191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370cfe27-bff3-4f1b-a146-c786e64bde88 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.263730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.264348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.264623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.264969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.265625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:32.265932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.267698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.283724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1045/2099] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:32.289149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.302144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5a45cc-7ace-46be-bb44-8388343b809e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.309178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.310001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.310343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.310827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.311503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.313567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.345868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1055/2100] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:32 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:32.350289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.361004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b62069-9840-48e9-9206-1476f826aedd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.428783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.429404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.429683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.429958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.430607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.432598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.457897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1046/2101] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:32 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:32.462372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.469701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:32.474088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7185834-2489-4a0f-afc8-a79d39734839 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.481414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.482051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.482498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.482959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.483764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.485612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.498987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1056/2102] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:32.503261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.514494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae69432d-b0b6-4d8d-9711-60fe621b2d94 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.521019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.521641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.521907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.522171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.522830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.524521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b846ce9-db77-4c71-b242-f8af2d7277bf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.536321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1047/2103] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:32.542548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.556926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d171df-60aa-4cf4-8f79-641b8bd8a004 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.564791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.565871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.566296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.566738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] PATH_INFO: `/v3/projects/bade80461cf849678b7508b05436c13b/users/5050f248719d4658aa3ed5eecc52a366/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.591228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bade80461cf849678b7508b05436c13b, user_id=5050f248719d4658aa3ed5eecc52a366, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.595325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.612855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1057/2104] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:32 2022] PUT /identity/v3/projects/bade80461cf849678b7508b05436c13b/users/5050f248719d4658aa3ed5eecc52a366/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:32.626506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.627009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.627538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.673766 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] MFA Rules not processed for user `5050f248719d4658aa3ed5eecc52a366`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:32.675223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:32.749806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1048/2105] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:32.761721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:32.763805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:32.863346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb07e77-8933-4aa8-9d30-fcffd72a5e85 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:32.867203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:32.867819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:32.868092 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:32.868367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:32.877547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:32.879010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade38da8-a798-4b8c-a097-e92982765c63 service glance] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:32.889500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1058/2106] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:32 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:40.195395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:40.197074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:40.252957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:40.276373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fae7e8b-c84a-43a7-a974-dc9488295a3e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:40.281840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:40.282604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:40.282980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:40.283411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] PATH_INFO: `/v3/users/b68f03248ac84002899e496286e9a24f` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:40.284782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b68f03248ac84002899e496286e9a24f)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:40.286940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:40.344351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b68f03248ac84002899e496286e9a24f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.351678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b68f03248ac84002899e496286e9a24f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.361032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b68f03248ac84002899e496286e9a24f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.361661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b68f03248ac84002899e496286e9a24f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.362771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b68f03248ac84002899e496286e9a24f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.385123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1049/2107] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:40 2022] DELETE /identity/v3/users/b68f03248ac84002899e496286e9a24f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:40.389914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:40.390560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:40.390560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:40.410816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:40.412107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:40.474416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1059/2108] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:40.486549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:40.488441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:40.549804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07259d37-f4bd-4565-a643-4af68c63cdcc admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:40.554279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:40.554912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:40.555239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:40.555514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:40.566052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:40.566990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4068fe0d-66c2-41a0-b2ff-28dfbee69466 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:40.577019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1050/2109] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:40.630767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:40.632526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:40.648403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b403c1-259e-4a94-90a9-d28ce65c180f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:40.653934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:40.654829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:40.655277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:40.655652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] PATH_INFO: `/v3/users/ee51b102e38347d6a613bf15f9e63572` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:40.657018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee51b102e38347d6a613bf15f9e63572)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:40.659164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:40.756397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee51b102e38347d6a613bf15f9e63572'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.757761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:40.772311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee51b102e38347d6a613bf15f9e63572'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.773645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee51b102e38347d6a613bf15f9e63572'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.774606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee51b102e38347d6a613bf15f9e63572'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.794377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee51b102e38347d6a613bf15f9e63572'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:40.868795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1060/2110] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:40 2022] DELETE /identity/v3/users/ee51b102e38347d6a613bf15f9e63572 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:40.870814 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:40.870814 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:40.871186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:40.919270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:40.920560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.050968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:41.052838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.065171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1051/2111] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:41.090208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:41.091787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.114206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:41.160099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d57be-8cc4-49fe-b236-4e7de6d4646f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:41.165558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:41.166197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.166473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.166746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] PATH_INFO: `/v3/projects/8a9e3b846df943e0838de5bc7c31b418` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.168476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9e3b846df943e0838de5bc7c31b418)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:41.170471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:41.172755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbcf39-7ddf-4914-aff9-f5031d396245 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:41.177173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:41.177805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.178088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.178363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.189558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:41.191106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:41.201445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1052/2112] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:37:41.227405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9e3b846df943e0838de5bc7c31b418', 'request_id': 'req-361153aa-26a2-4306-975a-d5c44b119950'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.233818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9e3b846df943e0838de5bc7c31b418', 'request_id': 'req-361153aa-26a2-4306-975a-d5c44b119950'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.245944 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9e3b846df943e0838de5bc7c31b418', 'request_id': 'req-361153aa-26a2-4306-975a-d5c44b119950'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.258795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] The token cache is being invalidate because project 8a9e3b846df943e0838de5bc7c31b418 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:41.259012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.259714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.261593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1061/2113] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:41 2022] DELETE /identity/v3/projects/8a9e3b846df943e0838de5bc7c31b418 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:41.266379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:41.279988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818b19e-4de7-4c3c-ab9e-be8f309f412c service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:41.346354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:41.347099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.347500 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.347773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] PATH_INFO: `/v3/users/843f70e76d704de98a30497c2d1ed3e6` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.350027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=843f70e76d704de98a30497c2d1ed3e6)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:41.352878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:41.419023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '843f70e76d704de98a30497c2d1ed3e6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.426548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '843f70e76d704de98a30497c2d1ed3e6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.435351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '843f70e76d704de98a30497c2d1ed3e6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.436110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '843f70e76d704de98a30497c2d1ed3e6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.436760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-122fde4e-9d35-446e-aca0-54b61b735e75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '843f70e76d704de98a30497c2d1ed3e6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.454951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1053/2114] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:41 2022] DELETE /identity/v3/users/843f70e76d704de98a30497c2d1ed3e6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:41.517398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:41.519025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.544911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361153aa-26a2-4306-975a-d5c44b119950 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:41.614876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:41.631764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:41.632277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.633156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.633498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] PATH_INFO: `/v3/projects/abd21af5094241d68fe66d1a9a879ec3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.634918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd21af5094241d68fe66d1a9a879ec3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:41.637200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:41.708653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd21af5094241d68fe66d1a9a879ec3', 'request_id': 'req-727bd358-2f2d-4167-844a-3b8a93730f3b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.715093 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd21af5094241d68fe66d1a9a879ec3', 'request_id': 'req-727bd358-2f2d-4167-844a-3b8a93730f3b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.724313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd21af5094241d68fe66d1a9a879ec3', 'request_id': 'req-727bd358-2f2d-4167-844a-3b8a93730f3b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.738599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] The token cache is being invalidate because project abd21af5094241d68fe66d1a9a879ec3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:41.738767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.739441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:41.741425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1062/2115] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:41 2022] DELETE /identity/v3/projects/abd21af5094241d68fe66d1a9a879ec3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:41.814389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3b34c-f29b-4815-911f-de45d4016840 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.814389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3b34c-f29b-4815-911f-de45d4016840 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.814675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b3b34c-f29b-4815-911f-de45d4016840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.822846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:41.825710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.859836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e6b3b34c-f29b-4815-911f-de45d4016840 None None] MFA Rules not processed for user `bd8f3afb4bf040c58feb266b45c0f503`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:41.864041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e6b3b34c-f29b-4815-911f-de45d4016840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:41.927002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:41.980453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bd358-2f2d-4167-844a-3b8a93730f3b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:41.994692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:41.995928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:41.995928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:41.995928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] PATH_INFO: `/v3/projects/0e5706b8531a4fbdb091069bec143a11` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:41.997047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5706b8531a4fbdb091069bec143a11)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:41.999066 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.007967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1054/2116] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:42.073388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5706b8531a4fbdb091069bec143a11', 'request_id': 'req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:42.080166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5706b8531a4fbdb091069bec143a11', 'request_id': 'req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:42.090440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5706b8531a4fbdb091069bec143a11', 'request_id': 'req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:42.113850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] The token cache is being invalidate because project 0e5706b8531a4fbdb091069bec143a11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:42.114229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:42.115174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-dcbcd75c-4eb6-4e46-9747-5481aa36f5a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:42.116749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1063/2117] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:41 2022] DELETE /identity/v3/projects/0e5706b8531a4fbdb091069bec143a11 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:42.127168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.127168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.127168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.150493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:42.152031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:42.225121 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1055/2118] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:42.233964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.234350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.234633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.257466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:42.258419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:42.275235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1064/2119] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:42.276182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.286196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b614166-fd69-41a6-9558-25eb6c2dfa25 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.291561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.292203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.292483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.292775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.293498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:42.293756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.295467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.311786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1056/2120] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:37:42.313241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.323040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec7129-edd2-40d9-a54a-d5c020d19f07 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.332344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.333120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.333400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.333649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.334286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.336276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.342251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:42.372484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1065/2121] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:42.376756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.389219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bc1ee-de46-4b93-978b-8d7d6c1a6c7b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.454469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.455105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.455415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.455684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.456379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.458314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.480401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1057/2122] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/users => generated 297 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:37:42.490798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.502172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f33d0d7-cabb-4e43-a04f-5bb82d997c53 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.508320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.508951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.509249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.509521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.510203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.511872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.522911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1066/2123] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:37:42.527063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.537571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f12048-2c10-4df7-a649-4b87d959c317 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.544272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.544833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.545178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.545366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.546028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.547725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b323d8-356b-4e5f-9251-955dc055e0c4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.562123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1058/2124] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:42.563581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.575365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5f666-a7bd-4795-b314-38e36d98ac10 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.581361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.582063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.582339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.582607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] PATH_INFO: `/v3/projects/622c3cb0d25e4ffd8ea6d59db6801446/users/80e51f9fe32e42d18dfad1c44c52ba66/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.600667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=622c3cb0d25e4ffd8ea6d59db6801446, user_id=80e51f9fe32e42d18dfad1c44c52ba66, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.603558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.618210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1067/2125] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:42 2022] PUT /identity/v3/projects/622c3cb0d25e4ffd8ea6d59db6801446/users/80e51f9fe32e42d18dfad1c44c52ba66/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:42.630951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.631303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.631569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.663301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] MFA Rules not processed for user `80e51f9fe32e42d18dfad1c44c52ba66`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:42.664608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:42.706249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:42.756902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1059/2126] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:42.764222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.840323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928da7b5-c4b1-4341-9f94-2f74b26ecb1a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.845185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.845794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.846069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.846335 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.846964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.848965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:42.881441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1068/2127] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/projects => generated 391 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:42.883354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:42.899479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d22dd9-a4bf-474b-ae3e-8c91578644df None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:42.968170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:42.969032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:42.969452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:42.969841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:42.970581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:42.972669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.000088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1060/2128] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:42 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:43.004235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.016662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee9db3-ae1e-4b0b-864b-f66f4ec9ba74 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.023617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.024361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.024723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.025048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.025884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.027734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.044779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1069/2129] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:37:43.050956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.061832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3def37-6ca3-4b45-b2fc-5af34baf130a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.067596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.068455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.068935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.069281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] PATH_INFO: `/v3/projects/f84561088ba04d3eafa59a974ce3c7e8/users/f280b3ca306542359d1ecb7e940bd846/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.087812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84561088ba04d3eafa59a974ce3c7e8, user_id=f280b3ca306542359d1ecb7e940bd846, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.090508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.104386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1061/2130] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:43 2022] PUT /identity/v3/projects/f84561088ba04d3eafa59a974ce3c7e8/users/f280b3ca306542359d1ecb7e940bd846/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:43.109618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.120417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cb534-5838-44ca-b413-0b9a82e38995 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.179518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.180135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.180410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.180683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.181330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.182954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f066b0-9f74-498a-abb9-3e293e5bc5fc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.194914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1070/2131] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:43.199395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.212590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02936e85-3fcf-497a-ad18-b25e0ee35afe admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.215249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.215937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.216222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.216507 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] PATH_INFO: `/v3/projects/f84561088ba04d3eafa59a974ce3c7e8/users/f280b3ca306542359d1ecb7e940bd846/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.218667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84561088ba04d3eafa59a974ce3c7e8, user_id=f280b3ca306542359d1ecb7e940bd846, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.221372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.233524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1062/2132] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:43 2022] PUT /identity/v3/projects/f84561088ba04d3eafa59a974ce3c7e8/users/f280b3ca306542359d1ecb7e940bd846/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:43.244577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.244860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.245179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.281817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] MFA Rules not processed for user `f280b3ca306542359d1ecb7e940bd846`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:43.283636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:43.303043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:43.389520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1071/2133] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:43 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:43.404597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.406904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:43.473937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f4d59-46a0-4c55-93fc-94e929d15dbe admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.478152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.478787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.479266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.479752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.491578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.493179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.505202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1063/2134] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:43 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:43.556263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:43.557784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:43.571753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec82473-2861-47c2-9232-f877b5ee27cb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:43.577048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:43.577698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:43.578000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:43.578259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:43.643366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:43.645042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86dd5a8-273f-4019-b986-7b5ffdb56249 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:43.654946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1072/2135] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:47.249223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:47.282792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:47.328498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8937b-1aa0-43ca-a628-0413f50971f7 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:47.334057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:47.334712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:47.334992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:47.335301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] PATH_INFO: `/v3/users/80e51f9fe32e42d18dfad1c44c52ba66` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:47.336463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e51f9fe32e42d18dfad1c44c52ba66)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:47.338474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:47.408639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e51f9fe32e42d18dfad1c44c52ba66'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:47.416236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e51f9fe32e42d18dfad1c44c52ba66'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:47.425872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e51f9fe32e42d18dfad1c44c52ba66'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:47.426650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e51f9fe32e42d18dfad1c44c52ba66'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:47.427495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e51f9fe32e42d18dfad1c44c52ba66'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:47.447178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1064/2136] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:47 2022] DELETE /identity/v3/users/80e51f9fe32e42d18dfad1c44c52ba66 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:47.452415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:47.452978 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:47.453380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:47.473168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:47.474467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:47.545139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1073/2137] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:47.556387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:47.558037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:47.629851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60b03b-f0cd-4a98-aee9-8b27cfe0b07c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:47.635622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:47.636496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:47.636775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:47.637091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:47.648330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:47.649843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:47.659997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1065/2138] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:47.969207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:47.970722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:47.985289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab8e3f-768a-4361-9686-ac63abc2b931 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:47.990888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:47.993506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:47.993506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:47.993506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] PATH_INFO: `/v3/projects/622c3cb0d25e4ffd8ea6d59db6801446` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:47.994894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=622c3cb0d25e4ffd8ea6d59db6801446)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:47.998661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:48.044554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:48.056427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '622c3cb0d25e4ffd8ea6d59db6801446', 'request_id': 'req-d8bfc863-7e87-4a32-84a9-9b48d342f045'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.069452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '622c3cb0d25e4ffd8ea6d59db6801446', 'request_id': 'req-d8bfc863-7e87-4a32-84a9-9b48d342f045'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.078184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '622c3cb0d25e4ffd8ea6d59db6801446', 'request_id': 'req-d8bfc863-7e87-4a32-84a9-9b48d342f045'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.091841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] The token cache is being invalidate because project 622c3cb0d25e4ffd8ea6d59db6801446 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:48.092127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.092896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.094869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1074/2139] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:47 2022] DELETE /identity/v3/projects/622c3cb0d25e4ffd8ea6d59db6801446 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:48.100089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:48.117142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee73c8d-1fed-4052-905b-135d97e50d7e service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:48.171000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:48.184764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:48.185419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:48.185707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:48.185983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] PATH_INFO: `/v3/users/f280b3ca306542359d1ecb7e940bd846` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:48.187193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f280b3ca306542359d1ecb7e940bd846)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:48.189261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:48.250207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f280b3ca306542359d1ecb7e940bd846'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.257958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f280b3ca306542359d1ecb7e940bd846'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.267180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f280b3ca306542359d1ecb7e940bd846'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.267831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f280b3ca306542359d1ecb7e940bd846'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.268482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-584b2972-890d-4af7-8c60-e7be40c54a02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f280b3ca306542359d1ecb7e940bd846'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.289475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1066/2140] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:37:48 2022] DELETE /identity/v3/users/f280b3ca306542359d1ecb7e940bd846 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:48.696090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:48.698426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:48.719402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bfc863-7e87-4a32-84a9-9b48d342f045 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:48.799997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:48.800657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:48.800938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:48.801213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] PATH_INFO: `/v3/projects/f84561088ba04d3eafa59a974ce3c7e8` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:48.802823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84561088ba04d3eafa59a974ce3c7e8)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:48.804833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:48.856260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84561088ba04d3eafa59a974ce3c7e8', 'request_id': 'req-7c7fd3fd-9c4d-4037-84af-bee393991652'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.863116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84561088ba04d3eafa59a974ce3c7e8', 'request_id': 'req-7c7fd3fd-9c4d-4037-84af-bee393991652'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.874120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84561088ba04d3eafa59a974ce3c7e8', 'request_id': 'req-7c7fd3fd-9c4d-4037-84af-bee393991652'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.886820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] The token cache is being invalidate because project f84561088ba04d3eafa59a974ce3c7e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:37:48.887133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.887912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c7fd3fd-9c4d-4037-84af-bee393991652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:37:48.889557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1075/2141] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:37:48 2022] DELETE /identity/v3/projects/f84561088ba04d3eafa59a974ce3c7e8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:48.894853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:48.895240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:48.895525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:48.917468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:48.918702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:48.980457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1067/2142] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:48.989740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:48.989998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:48.990281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.013460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:49.014714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:49.024312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1076/2143] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:49.028782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.039561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff955a-6bd0-4f21-8cbb-4e215aa87edd None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.046227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.047005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.047458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.047725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.048389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:37:49.048685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.050416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.065604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1068/2144] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:37:49.070821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.081188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbfe46b-2af0-4f51-80c5-d1f82f897abe None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.094052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.094052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.094052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.094052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.094583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.094583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.122517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1077/2145] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:49 2022] POST /identity/v3/projects => generated 407 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:49.130478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.142974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ec287-294e-40f0-9d68-0c7c69301331 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.203379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:49.210946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.211865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.212467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.213003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.214217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.216721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.251157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1069/2146] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:49 2022] POST /identity/v3/users => generated 304 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:49.254590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.266793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa169d-5a0a-4482-b08d-b4e4f688a1bf admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.272880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.274734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.275018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.275336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.276033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.278261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.290829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1078/2147] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:49.294558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.306703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c129595-0799-457a-8234-87bf80c529c8 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.311660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.312290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.312570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.312924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.313594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.315975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222be388-c52f-4e02-a23f-b26c4908db50 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.327399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1070/2148] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:49.331965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.343017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0fab2-c71a-4405-9a5c-261ea8e62dc8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.348056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.348737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.349019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.349288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] PATH_INFO: `/v3/projects/a9e6849fdf21440d95967b075e09d2b2/users/2698481b2ac04418af395d6642f11ab7/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.366999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9e6849fdf21440d95967b075e09d2b2, user_id=2698481b2ac04418af395d6642f11ab7, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.369712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.389486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:49.400564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1079/2149] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:49 2022] PUT /identity/v3/projects/a9e6849fdf21440d95967b075e09d2b2/users/2698481b2ac04418af395d6642f11ab7/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:49.414731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.415127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.415310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.451356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] MFA Rules not processed for user `2698481b2ac04418af395d6642f11ab7`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:49.452614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:49.525324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1071/2150] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:37:49.529976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.600499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e8b382-b2dc-49a3-a7d3-829eb9406388 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.605456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.606202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.606479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.606794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.607619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.609703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.641515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1080/2151] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:37:49 2022] POST /identity/v3/projects => generated 409 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:49.645450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.657806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcaf2ea-f56e-4b77-ad23-923b08487928 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.722207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.722850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.723146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.723487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.724110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.726475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.755348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1072/2152] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:37:49 2022] POST /identity/v3/users => generated 311 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:37:49.759317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.771279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca18909-9df9-4110-a0b6-72ebc60c5658 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.776767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.777645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.778125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.778293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.778893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.780586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.799185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1081/2153] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:49.802714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.813796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797eb67f-f79c-481b-a819-17e660ddcd1d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.821848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.822705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.822987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.823303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] PATH_INFO: `/v3/projects/8fe256776eaa4f8fae98719bcb4c034a/users/706290440fdf4f3fa69b3ff189efd786/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.841366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe256776eaa4f8fae98719bcb4c034a, user_id=706290440fdf4f3fa69b3ff189efd786, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.846097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.865133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1073/2154] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:49 2022] PUT /identity/v3/projects/8fe256776eaa4f8fae98719bcb4c034a/users/706290440fdf4f3fa69b3ff189efd786/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:37:49.869128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.881315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bec25-a88b-4ed6-9f07-358ed18c911c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.956812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.958298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.958298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.958560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:49.959226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:49.961030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a7e1b-13d1-4225-8a99-f45c71f291f2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:49.972903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1082/2155] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:37:49 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:37:49.976683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:49.988685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a9669-ed79-4904-aa91-5538beaeb41d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:49.996826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:49.997550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:49.998101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:49.998448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] PATH_INFO: `/v3/projects/8fe256776eaa4f8fae98719bcb4c034a/users/706290440fdf4f3fa69b3ff189efd786/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:50.000815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fe256776eaa4f8fae98719bcb4c034a, user_id=706290440fdf4f3fa69b3ff189efd786, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:50.004618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:50.019129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1074/2156] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:37:49 2022] PUT /identity/v3/projects/8fe256776eaa4f8fae98719bcb4c034a/users/706290440fdf4f3fa69b3ff189efd786/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:37:50.029714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:50.030014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:50.030272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:50.062523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] MFA Rules not processed for user `706290440fdf4f3fa69b3ff189efd786`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:37:50.064786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:50.122779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:50.146396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1083/2157] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:37:50 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:37:50.157696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:50.159670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:37:50.183272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:37:50.226251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe38e0-3120-42f3-a7d7-7a3537400249 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:50.230222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:50.230899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:50.231206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:50.231472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:50.241524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:50.242969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:50.253688 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1075/2158] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:50 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:37:50.443062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:37:50.456428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ca979-4d1d-4ad8-a7fe-4659aea78fd2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:37:50.460251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:37:50.460855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:37:50.461134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:37:50.461404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:37:50.529669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:37:50.531188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c98d4c-380d-484d-94ad-aad5b2ee83a7 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:37:50.541989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1084/2159] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:37:50 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:00.249128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=18 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:00.536983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=32 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:04.378922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:04.483549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702241c-6944-4da2-ab57-7e2c8fa205ba service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:04.493581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:04.494291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:04.494542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:04.494811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] PATH_INFO: `/v3/users/2698481b2ac04418af395d6642f11ab7` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:04.496011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2698481b2ac04418af395d6642f11ab7)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:04.498014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:04.557411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2698481b2ac04418af395d6642f11ab7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:04.570774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2698481b2ac04418af395d6642f11ab7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:04.578386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2698481b2ac04418af395d6642f11ab7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:04.579169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2698481b2ac04418af395d6642f11ab7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:04.579955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2698481b2ac04418af395d6642f11ab7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:04.600477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1076/2160] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:04 2022] DELETE /identity/v3/users/2698481b2ac04418af395d6642f11ab7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:04.605353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:04.605950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:04.605950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:04.626812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:04.628099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:04.692096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1085/2161] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:04.703122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:04.704886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:04.782829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede11f75-0b22-4d8d-8ee6-6f2e595f3f89 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:04.788690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:04.789780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:04.790488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:04.790757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:04.808138 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:04.809632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:04.824301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1077/2162] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:38:05.160750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:05.162574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:05.177158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830372de-8c8d-4a14-b766-7205146c4567 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:05.183035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:05.183855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:05.184694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:05.184694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] PATH_INFO: `/v3/projects/a9e6849fdf21440d95967b075e09d2b2` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:05.185789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9e6849fdf21440d95967b075e09d2b2)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:05.187804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:05.249541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9e6849fdf21440d95967b075e09d2b2', 'request_id': 'req-c94981a0-f965-4774-81ba-c291213d4270'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.256425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9e6849fdf21440d95967b075e09d2b2', 'request_id': 'req-c94981a0-f965-4774-81ba-c291213d4270'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.266216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9e6849fdf21440d95967b075e09d2b2', 'request_id': 'req-c94981a0-f965-4774-81ba-c291213d4270'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.280201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] The token cache is being invalidate because project a9e6849fdf21440d95967b075e09d2b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:05.280541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.281373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.283222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1086/2163] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:05 2022] DELETE /identity/v3/projects/a9e6849fdf21440d95967b075e09d2b2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:05.287564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:05.297975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a87684-4c45-4ab9-a641-772f1f512e82 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:05.320126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:05.382426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:05.383429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:05.383429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:05.383722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] PATH_INFO: `/v3/users/706290440fdf4f3fa69b3ff189efd786` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:05.385218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=706290440fdf4f3fa69b3ff189efd786)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:05.387308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:05.444186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '706290440fdf4f3fa69b3ff189efd786'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.465614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '706290440fdf4f3fa69b3ff189efd786'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.481448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '706290440fdf4f3fa69b3ff189efd786'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.482809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '706290440fdf4f3fa69b3ff189efd786'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.484025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c72ca1c-0605-4933-965f-1eb52a2f8ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '706290440fdf4f3fa69b3ff189efd786'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:05.515707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1078/2164] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:05 2022] DELETE /identity/v3/users/706290440fdf4f3fa69b3ff189efd786 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:06.191467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.193176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:06.211446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94981a0-f965-4774-81ba-c291213d4270 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.298946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.299856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.299856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.300240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] PATH_INFO: `/v3/projects/8fe256776eaa4f8fae98719bcb4c034a` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.301562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fe256776eaa4f8fae98719bcb4c034a)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.303631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.343615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:06.377599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fe256776eaa4f8fae98719bcb4c034a', 'request_id': 'req-a4e94f38-e304-4e12-ad8c-9ff96910dedc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:06.384317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fe256776eaa4f8fae98719bcb4c034a', 'request_id': 'req-a4e94f38-e304-4e12-ad8c-9ff96910dedc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:06.407747 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fe256776eaa4f8fae98719bcb4c034a', 'request_id': 'req-a4e94f38-e304-4e12-ad8c-9ff96910dedc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:06.414852 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] The token cache is being invalidate because project 8fe256776eaa4f8fae98719bcb4c034a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:06.415183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:06.415869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a4e94f38-e304-4e12-ad8c-9ff96910dedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:06.417736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1087/2165] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:06 2022] DELETE /identity/v3/projects/8fe256776eaa4f8fae98719bcb4c034a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:06.423733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.424390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.424390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.448109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:06.449468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:06.535606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1079/2166] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:06.544038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.544973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.545551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.587705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:06.589402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:06.603559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1088/2167] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:06.608776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.620655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc50eb45-05ec-4ab8-9623-195b2e292137 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.626509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.626754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.626940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.628188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.628188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:38:06.628459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.630631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.651094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1080/2168] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:38:06.655777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.665812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a17b4-7650-46ad-b808-121b15b0472c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.670820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.671559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.671831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.672387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.672760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.674688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.706568 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1089/2169] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:06 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:06.711657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.725741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2ae22-f9dc-42d3-8335-cfcf2e0617bf admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.785111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:06.836749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.837397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.837934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.838112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.838777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.841000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.885203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1081/2170] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:06 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:38:06.891742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.911128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c0674-007b-4fd8-b010-929540293558 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.930562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.931769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.933855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.933855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.934061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.937092 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.958837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1090/2171] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:06.962940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:06.974220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d71937-1c26-4e23-af03-7a9658a7ea51 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:06.980292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:06.980916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:06.981185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:06.981453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:06.982110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:06.983775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba1c46b-dcc3-44ec-8434-05baafdad3bf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:06.997264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1082/2172] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:38:07.001452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.012094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63438aa1-2e53-4208-9783-7d6bcc725436 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.016864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.017546 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.017810 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.018079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] PATH_INFO: `/v3/projects/f26637b19741498b82f5c098196e4921/users/2c90aa24b62d4c7cabdfee049efec794/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.035299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26637b19741498b82f5c098196e4921, user_id=2c90aa24b62d4c7cabdfee049efec794, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.037960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.059281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1091/2173] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:07 2022] PUT /identity/v3/projects/f26637b19741498b82f5c098196e4921/users/2c90aa24b62d4c7cabdfee049efec794/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:07.069938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.070278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.070495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.103557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] MFA Rules not processed for user `2c90aa24b62d4c7cabdfee049efec794`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:07.104675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:07.180037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1083/2174] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:07.184895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.282745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea1463b-b3db-4463-a955-ba417122953f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.291437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.292470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.292974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.293476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.294664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.298505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.343643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1092/2175] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/projects => generated 377 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:07.347906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.360025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35652d4b-a3e3-404e-b890-ee7091576f35 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.453193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.453952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.454108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.454326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.454980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.457110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.482397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1084/2176] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/users => generated 289 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:07.488084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.504958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc1cee-1394-45d2-a4d7-8cdd711a1a42 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.513169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.514257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.514742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.515275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.516469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.519680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.541008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1093/2177] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:07.545285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.556643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99acce41-ad1b-4480-8eb7-69ea72803790 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.562503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.563173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.563454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.563726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.564388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.566061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a4d96-e9eb-44fc-a89a-68a78332a1dc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.578147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1085/2178] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:38:07.583336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.593900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771b8430-8bd6-4ac9-a8a8-635dbcb9c03e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.598932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.599675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.600108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.600392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] PATH_INFO: `/v3/projects/572bb24609184aa48bfcda42ff61c84b/users/ae65b46db07f48af96afb35d0c4c8989/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.618664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=572bb24609184aa48bfcda42ff61c84b, user_id=ae65b46db07f48af96afb35d0c4c8989, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.623215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.636125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:07.642967 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1094/2179] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:07 2022] PUT /identity/v3/projects/572bb24609184aa48bfcda42ff61c84b/users/ae65b46db07f48af96afb35d0c4c8989/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:07.656747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.657100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.657440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.690064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] MFA Rules not processed for user `ae65b46db07f48af96afb35d0c4c8989`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:07.691394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:07.695827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:07.774501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1086/2180] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:07.779582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.862232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e45b942-11e9-45ea-a5c6-b52909ad2f04 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:07.870487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:07.871486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:07.871982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:07.872478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:07.873649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:07.877512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:07.920437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1095/2181] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/projects => generated 379 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:07.924985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:07.937234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1c84b-b228-42b7-a308-5f50e8b94f19 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.030805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.031624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.032008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.032333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.033021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.035223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.065105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1087/2182] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:07 2022] POST /identity/v3/users => generated 296 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:08.071834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.085572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29496039-eecf-409f-afc5-a46c89d0846c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.092150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.092900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.093238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.093537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.094314 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.096442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.116905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1096/2183] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:08.119042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.130631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1935acd-3598-4cc2-8cca-d7d31066d762 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.136374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.137710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.137710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.137710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] PATH_INFO: `/v3/projects/0f42c2380e164e5590d941592be6a2b3/users/f468cd9700594c4db3fe5691167e19d6/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.160108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f42c2380e164e5590d941592be6a2b3, user_id=f468cd9700594c4db3fe5691167e19d6, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.166324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.185445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1088/2184] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:08 2022] PUT /identity/v3/projects/0f42c2380e164e5590d941592be6a2b3/users/f468cd9700594c4db3fe5691167e19d6/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:08.190065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.202118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c497179-b7e0-40cc-b151-4a149ef52b31 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.269591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.270219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.270497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.270793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.271490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.273135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924f5937-9ca2-476a-87e9-4d2c49fca13c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.285261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1097/2185] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:08.290642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.302341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc34930-57b1-4d18-9479-30e31f25f1a6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.307498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.308183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.308456 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.308764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] PATH_INFO: `/v3/projects/0f42c2380e164e5590d941592be6a2b3/users/f468cd9700594c4db3fe5691167e19d6/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.312343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f42c2380e164e5590d941592be6a2b3, user_id=f468cd9700594c4db3fe5691167e19d6, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.315097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.330239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1089/2186] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:08 2022] PUT /identity/v3/projects/0f42c2380e164e5590d941592be6a2b3/users/f468cd9700594c4db3fe5691167e19d6/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:08.342637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.342923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.343247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.383713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] MFA Rules not processed for user `f468cd9700594c4db3fe5691167e19d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:08.385563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:08.477170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:08.496999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1098/2187] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:08 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:38:08.512777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.515919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:08.603190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8561fb76-8043-409f-8b4b-5da421d5d10a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.609233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.610289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.611204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.611668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.628769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.631583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6a471d-b2ab-42a7-97bf-4304a2600977 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.649809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1090/2188] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:08.650754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:08.696975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:08.698761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:08.712487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88826558-ecae-4332-abc1-226977c3eb0a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:08.716708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:08.717341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:08.717678 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:08.717908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:08.792709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:08.794177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:08.804405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1099/2189] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:18.799789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=30 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:22.731346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:22.731854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:22.731970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:22.760236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:22.761468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:22.823490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1091/2190] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:38:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:38:22.884395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:22.886139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:22.961981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0fce7-e876-4b61-9b05-ef2dacbb095f service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:22.969620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:22.970533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:22.970855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:22.971147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:22.982599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:22.984274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeaeff9-b7d4-4835-827b-edb6e13c1dae service swift] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:22.996005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1100/2191] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:38:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 13:38:23.591840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:23.786631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d522be-044a-41ff-acab-1c186ad645da None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:23.830192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:23.830957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:23.831290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:23.843888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] PATH_INFO: `/v3/users/2c90aa24b62d4c7cabdfee049efec794` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:23.863835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c90aa24b62d4c7cabdfee049efec794)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:23.863835 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:23.986302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c90aa24b62d4c7cabdfee049efec794'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:23.996157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c90aa24b62d4c7cabdfee049efec794'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:24.007993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c90aa24b62d4c7cabdfee049efec794'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:24.008794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c90aa24b62d4c7cabdfee049efec794'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:24.010126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c90aa24b62d4c7cabdfee049efec794'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:24.033107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1092/2192] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:23 2022] DELETE /identity/v3/users/2c90aa24b62d4c7cabdfee049efec794 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:24.037393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:24.037956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:24.037956 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:24.059619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:24.060947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:24.170845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1101/2193] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:38:24.179705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:24.182222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:24.226182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:24.297004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f7311-e8f2-49dd-9622-87e242cfc69f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:24.304095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:24.304838 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:24.305242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:24.307754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:24.318775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:24.320582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:24.347761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1093/2194] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:25.123722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:25.125244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:25.149959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ae0ed-7fe6-4038-9b3a-900c95756ace None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:25.158254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:25.159169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:25.160638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:25.161334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] PATH_INFO: `/v3/projects/f26637b19741498b82f5c098196e4921` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:25.164266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26637b19741498b82f5c098196e4921)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:25.170198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:25.243188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26637b19741498b82f5c098196e4921', 'request_id': 'req-6836d5df-3dd0-41bb-893a-3a18925eccc9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.251626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26637b19741498b82f5c098196e4921', 'request_id': 'req-6836d5df-3dd0-41bb-893a-3a18925eccc9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.262933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26637b19741498b82f5c098196e4921', 'request_id': 'req-6836d5df-3dd0-41bb-893a-3a18925eccc9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.281431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] The token cache is being invalidate because project f26637b19741498b82f5c098196e4921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:25.282227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.283157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.288008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1102/2195] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:25 2022] DELETE /identity/v3/projects/f26637b19741498b82f5c098196e4921 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:25.289246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:25.301862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f647b-a4a1-43c4-aca0-8910958afba8 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:25.369365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:25.369933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:25.370215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:25.370491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] PATH_INFO: `/v3/users/ae65b46db07f48af96afb35d0c4c8989` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:25.371833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae65b46db07f48af96afb35d0c4c8989)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:25.374044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:25.458848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae65b46db07f48af96afb35d0c4c8989'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.472750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae65b46db07f48af96afb35d0c4c8989'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.490066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae65b46db07f48af96afb35d0c4c8989'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.491807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae65b46db07f48af96afb35d0c4c8989'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.493255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-71cc7125-7382-468d-ac2e-6a0c37540877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae65b46db07f48af96afb35d0c4c8989'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:25.522657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1094/2196] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:25 2022] DELETE /identity/v3/users/ae65b46db07f48af96afb35d0c4c8989 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:26.058797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:26.060297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:26.081220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836d5df-3dd0-41bb-893a-3a18925eccc9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:26.137466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:26.185473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:26.185957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:26.186333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:26.221551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] MFA Rules not processed for user `518d6948777944fd8214dc6d778a9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:26.222927 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:26.223358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:26.223833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:26.224218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:26.224593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] PATH_INFO: `/v3/projects/572bb24609184aa48bfcda42ff61c84b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:26.225948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] RBAC: Authorizing `identity:delete_project(project_id=572bb24609184aa48bfcda42ff61c84b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:26.228778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:26.290667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '572bb24609184aa48bfcda42ff61c84b', 'request_id': 'req-cc617466-63b9-422a-82a6-c2562418827c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.299670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '572bb24609184aa48bfcda42ff61c84b', 'request_id': 'req-cc617466-63b9-422a-82a6-c2562418827c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.304878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:26.314105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '572bb24609184aa48bfcda42ff61c84b', 'request_id': 'req-cc617466-63b9-422a-82a6-c2562418827c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.331679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] The token cache is being invalidate because project 572bb24609184aa48bfcda42ff61c84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:26.331970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.332667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.335137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1095/2197] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:38:26.335312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1103/2198] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:26 2022] DELETE /identity/v3/projects/572bb24609184aa48bfcda42ff61c84b => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:26.342378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:26.344260 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:26.366889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548bf59a-1984-41d9-8e0a-3df4b16b97bc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:26.463110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:26.463110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:26.463448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:26.463690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] PATH_INFO: `/v3/users/f468cd9700594c4db3fe5691167e19d6` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:26.466428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f468cd9700594c4db3fe5691167e19d6)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:26.469270 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:26.565661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f468cd9700594c4db3fe5691167e19d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.574227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f468cd9700594c4db3fe5691167e19d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.588182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f468cd9700594c4db3fe5691167e19d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.589301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f468cd9700594c4db3fe5691167e19d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.590014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f1e63cba-0b49-4113-9899-5d860d0d9a07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f468cd9700594c4db3fe5691167e19d6'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:26.628594 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1096/2199] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:26 2022] DELETE /identity/v3/users/f468cd9700594c4db3fe5691167e19d6 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:27.018727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.020481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:27.049097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc617466-63b9-422a-82a6-c2562418827c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.170585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.171872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.171872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.172322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] PATH_INFO: `/v3/projects/0f42c2380e164e5590d941592be6a2b3` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.173954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f42c2380e164e5590d941592be6a2b3)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.176780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:27.270618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:27.280950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f42c2380e164e5590d941592be6a2b3', 'request_id': 'req-6537abec-d57b-4ce9-8995-6b5d522d4f4c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:27.298690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f42c2380e164e5590d941592be6a2b3', 'request_id': 'req-6537abec-d57b-4ce9-8995-6b5d522d4f4c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:27.321398 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f42c2380e164e5590d941592be6a2b3', 'request_id': 'req-6537abec-d57b-4ce9-8995-6b5d522d4f4c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:27.348427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] The token cache is being invalidate because project 0f42c2380e164e5590d941592be6a2b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:27.348743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:27.350574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6537abec-d57b-4ce9-8995-6b5d522d4f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:27.352868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1104/2200] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:27 2022] DELETE /identity/v3/projects/0f42c2380e164e5590d941592be6a2b3 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:27.360879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.362094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.362713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.395364 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:27.397048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:27.529955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1097/2201] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:27.543330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6299-6210-4629-b390-53d4102409ba None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.543618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6299-6210-4629-b390-53d4102409ba None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.543894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e6299-6210-4629-b390-53d4102409ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.568561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c71e6299-6210-4629-b390-53d4102409ba None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:27.569825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c71e6299-6210-4629-b390-53d4102409ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:27.582435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1105/2202] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:27.583704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.601360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf2952-896b-4a88-a8d4-85236ad94e29 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.610897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.611939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.612296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.612636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.613966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:38:27.614742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.621606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:27.635846 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:27.667621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1098/2203] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:38:27.669086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e6299-6210-4629-b390-53d4102409ba None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.694698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e6299-6210-4629-b390-53d4102409ba None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.711764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.713238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.714165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.715955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.718118 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.722373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:27.775795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1106/2204] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:27 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:27.781338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.795618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b1830d-e147-4360-aee3-0edf57738760 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.881663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.882323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.882597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.882919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.883629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.885897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:27.915640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1099/2205] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:27 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:38:27.921046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.932589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190512e9-297e-4ad5-ba76-003e13fbb45c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.937386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.937968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.938295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.938549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.939278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.940959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:27.952634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1107/2206] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:27.957287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:27.968932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432dad7-8979-43c6-b2ec-e70970bc270f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:27.978742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:27.979563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:27.980058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:27.980526 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:27.981693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:27.984401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d48fe-a1eb-4263-b36a-5ae5d3ab5791 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:28.015415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1100/2207] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:27 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:28.016247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:28.030378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaec7a8-3bb4-4299-a305-f14754106c62 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:28.037727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:28.038577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.038925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.039355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] PATH_INFO: `/v3/projects/3a0f126a2b1e43a398aa0c8f49b25e04/users/9f8da76d28fe4f4691c0d65d6dd81347/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.069038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a0f126a2b1e43a398aa0c8f49b25e04, user_id=9f8da76d28fe4f4691c0d65d6dd81347, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:28.073603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:28.097850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1108/2208] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:28 2022] PUT /identity/v3/projects/3a0f126a2b1e43a398aa0c8f49b25e04/users/9f8da76d28fe4f4691c0d65d6dd81347/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:28.115651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59948477-7653-4204-8072-7f1110ed2476 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.116213 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59948477-7653-4204-8072-7f1110ed2476 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.116636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59948477-7653-4204-8072-7f1110ed2476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.157356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-59948477-7653-4204-8072-7f1110ed2476 None None] MFA Rules not processed for user `9f8da76d28fe4f4691c0d65d6dd81347`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:28.158579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-59948477-7653-4204-8072-7f1110ed2476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:28.254330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1101/2209] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:28.268627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:28.270362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:28.343408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0691b0-e82b-4460-b386-ce9c1a563b9a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:28.350374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:28.351218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.351500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.351848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.362853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:28.364427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a38dfc-e5e3-4a0c-89fb-9ae54347757e service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:28.405374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1109/2210] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:38:28.419946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59948477-7653-4204-8072-7f1110ed2476 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:28.514441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59948477-7653-4204-8072-7f1110ed2476 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:28.523707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:28.524396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.524682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.524943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] PATH_INFO: `/v3/users/9f8da76d28fe4f4691c0d65d6dd81347` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.526226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8da76d28fe4f4691c0d65d6dd81347)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:28.528246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:28.538540 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:28.612944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8da76d28fe4f4691c0d65d6dd81347'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:28.626922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8da76d28fe4f4691c0d65d6dd81347'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:28.636151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8da76d28fe4f4691c0d65d6dd81347'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:28.636782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8da76d28fe4f4691c0d65d6dd81347'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:28.639723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8da76d28fe4f4691c0d65d6dd81347'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:28.664233 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1102/2211] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:28 2022] DELETE /identity/v3/users/9f8da76d28fe4f4691c0d65d6dd81347 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:28.668576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.669051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.669329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.691794 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:28.693027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:28.779262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:28.780790 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1110/2212] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:28.781001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:28.855235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a3a186-b8b1-4aef-8b8d-37e824fc4968 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:28.862186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:28.862907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:28.863338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:28.863620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:28.874669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:28.876337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:28.889424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1103/2213] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:38:29.288811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:29.291287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:29.308050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897a31a-777e-4152-a4a9-d2ae550c21ee None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:29.327585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:29.328354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:29.328636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:29.328919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] PATH_INFO: `/v3/projects/3a0f126a2b1e43a398aa0c8f49b25e04` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:29.330212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a0f126a2b1e43a398aa0c8f49b25e04)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:29.332333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:29.343615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:29.474672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a0f126a2b1e43a398aa0c8f49b25e04', 'request_id': 'req-7d2cf96f-1323-4bc5-bc64-03a498974ce7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:29.502218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a0f126a2b1e43a398aa0c8f49b25e04', 'request_id': 'req-7d2cf96f-1323-4bc5-bc64-03a498974ce7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:29.513674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a0f126a2b1e43a398aa0c8f49b25e04', 'request_id': 'req-7d2cf96f-1323-4bc5-bc64-03a498974ce7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:29.532228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] The token cache is being invalidate because project 3a0f126a2b1e43a398aa0c8f49b25e04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:29.532228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:29.532710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:29.547111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1111/2214] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:29 2022] DELETE /identity/v3/projects/3a0f126a2b1e43a398aa0c8f49b25e04 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:29.692563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:29.699598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:29.831107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52bf86-aed0-428f-bdde-15cbba5df610 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:29.872168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:29.924593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:29.925681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:29.926177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:29.926673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:29.929807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:29.951669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:29.953331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41173b6d-8aff-4b5d-8932-93cde49322de service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:29.966447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1104/2215] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:36.685651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:36.687497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:36.792340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cf96f-1323-4bc5-bc64-03a498974ce7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:36.802402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:36.804152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:36.805020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:36.805645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] PATH_INFO: `/v3/users/5050f248719d4658aa3ed5eecc52a366` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:36.808547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5050f248719d4658aa3ed5eecc52a366)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:36.812781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:36.914346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5050f248719d4658aa3ed5eecc52a366'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:36.925769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5050f248719d4658aa3ed5eecc52a366'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:36.926737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5050f248719d4658aa3ed5eecc52a366'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:36.927575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5050f248719d4658aa3ed5eecc52a366'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:36.940491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5050f248719d4658aa3ed5eecc52a366'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:36.977212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1112/2216] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:36 2022] DELETE /identity/v3/users/5050f248719d4658aa3ed5eecc52a366 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:36.979952 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:36.980519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:36.980938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:37.016085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:37.018906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:37.096729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1105/2217] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:37.118685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:37.120466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:37.159905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:37.244737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf80857-db6b-4158-819d-f50d6d9c2f24 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:37.253293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:37.255300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:37.256052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:37.256777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:37.272861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:37.277807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83830199-0557-4742-848f-66520e3697f4 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:37.291012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1113/2218] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:38.021920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.026327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:38.081618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e2103-a198-4cd7-b322-63dc5af037d5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:38.091574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:38.093114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.093836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.095045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] PATH_INFO: `/v3/projects/bade80461cf849678b7508b05436c13b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.098297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bade80461cf849678b7508b05436c13b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:38.103949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:38.201621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bade80461cf849678b7508b05436c13b', 'request_id': 'req-3c667364-e04c-4e50-ae46-8b9482f4deff'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:38.210005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bade80461cf849678b7508b05436c13b', 'request_id': 'req-3c667364-e04c-4e50-ae46-8b9482f4deff'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:38.222027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bade80461cf849678b7508b05436c13b', 'request_id': 'req-3c667364-e04c-4e50-ae46-8b9482f4deff'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:38.239818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] The token cache is being invalidate because project bade80461cf849678b7508b05436c13b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:38.240618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:38.241849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3c667364-e04c-4e50-ae46-8b9482f4deff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:38.247662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1106/2219] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:38 2022] DELETE /identity/v3/projects/bade80461cf849678b7508b05436c13b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:38.264112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.264695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.264832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.332053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:38.334878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:38.498143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1114/2220] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:38.504348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.505245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.505853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.565509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:38.567681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:38.592296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1107/2221] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:38.593681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.606470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ff26a-03a0-4f8f-94b5-f9006bd2e880 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:38.618067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:38.618722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.619003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.619352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.620002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:38:38.620322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:38.622005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:38.655400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1115/2222] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:38:38.665365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.693616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fbc4b-4f28-49a2-84ae-23ff72966c38 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:38.703047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:38.704344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.704786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.705065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.705978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:38.710277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:38.759644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1108/2223] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:38 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:38:38.763772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.776705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a331b105-a650-44c0-aba9-09fe6cecf275 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:38.882525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:38.883308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.883594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.883845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.884544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:38.886759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:38.913726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1116/2224] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:38 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:38.918498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.937237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624fd32-9efd-4fb8-894d-fe93d03e4b75 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:38.942430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:38.943119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:38.943400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:38.943740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:38.944412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:38.946080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:38.965486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1109/2225] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:38.970940 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:38.994423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f62618d-ebb0-4d00-ba28-948a4a2ae4b2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:39.002871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:39.004144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:39.004700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:39.005263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:39.006623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:39.008917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4024ad4-b9b2-4380-9694-7b0f9286977a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:39.034828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1117/2226] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:38 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:39.039425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:39.046229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:39.050641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00240dc6-c83a-4c45-830f-8195c9246d0a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:39.063588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:39.070999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:39.071765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:39.072259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] PATH_INFO: `/v3/projects/25493e78c6c348ceb4f45faf2c4f9dd0/users/fcb6b58e1f804ca7aa86cfe87fc8c3c7/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:39.096470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25493e78c6c348ceb4f45faf2c4f9dd0, user_id=fcb6b58e1f804ca7aa86cfe87fc8c3c7, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:39.101864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:39.131129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1110/2227] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:39 2022] PUT /identity/v3/projects/25493e78c6c348ceb4f45faf2c4f9dd0/users/fcb6b58e1f804ca7aa86cfe87fc8c3c7/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:38:39.218701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8241ef-4d08-415f-bc60-6ea3547a240c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:39.221139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8241ef-4d08-415f-bc60-6ea3547a240c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:39.221715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8241ef-4d08-415f-bc60-6ea3547a240c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:39.272657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2b8241ef-4d08-415f-bc60-6ea3547a240c None None] MFA Rules not processed for user `fcb6b58e1f804ca7aa86cfe87fc8c3c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:39.274073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2b8241ef-4d08-415f-bc60-6ea3547a240c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:39.292732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:39.410562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1118/2228] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:39.424912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:39.427365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:39.509314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e47c57-7ea2-480a-9361-9d8993e93399 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:39.515720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:39.516384 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:39.516661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:39.516939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:39.526841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:39.528768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:39.546373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1111/2229] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:39 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:42.477601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:42.478123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:42.478123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:42.498801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:42.500062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:42.565705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1119/2230] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:38:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:42.582942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:42.584717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:42.666878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5128048c-a4e3-4c33-8644-4b0999789202 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:42.675113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:42.675925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:42.676293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:42.676628 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:42.687271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:42.688897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:42.697926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1112/2231] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:38:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 13:38:48.013968 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.016218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:48.097341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c13da2c-c70e-44b1-adeb-1b9a93f0279e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.102010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.102656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.102937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.103254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] PATH_INFO: `/v3/users/bd8f3afb4bf040c58feb266b45c0f503` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.104606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd8f3afb4bf040c58feb266b45c0f503)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.106597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.155962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd8f3afb4bf040c58feb266b45c0f503'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.163542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd8f3afb4bf040c58feb266b45c0f503'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.164236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd8f3afb4bf040c58feb266b45c0f503'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.164849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd8f3afb4bf040c58feb266b45c0f503'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.169111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:48.174522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-49de19c9-87c3-45a8-85ae-5e4e7ee795fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd8f3afb4bf040c58feb266b45c0f503'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.193982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1120/2232] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:48 2022] DELETE /identity/v3/users/bd8f3afb4bf040c58feb266b45c0f503 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:48.385840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.396692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7e5043-da77-4b7e-87ff-4caf5ec91d56 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.461141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.461774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.462067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.462317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] PATH_INFO: `/v3/projects/830ccc10c7284fc2942b49cd091e942a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.463601 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=830ccc10c7284fc2942b49cd091e942a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.465557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.483101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:48.517537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '830ccc10c7284fc2942b49cd091e942a', 'request_id': 'req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.523126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '830ccc10c7284fc2942b49cd091e942a', 'request_id': 'req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.535363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '830ccc10c7284fc2942b49cd091e942a', 'request_id': 'req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.550015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] The token cache is being invalidate because project 830ccc10c7284fc2942b49cd091e942a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:38:48.550673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.551762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bdddbb8-becf-4bb7-89ab-06a95d52ac49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:48.555099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1113/2233] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:38:48 2022] DELETE /identity/v3/projects/830ccc10c7284fc2942b49cd091e942a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:48.563515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.564159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.564545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.586021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:48.587322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:48.656387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1121/2234] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:48.666990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.669845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.670216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.701821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:48.703360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:48.714986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1114/2235] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:48.716971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.739329 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ecd25-3460-47bc-a16c-675d5f921f82 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.745007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.745635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.745912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.746210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.746846 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:38:48.747171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.748939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.762865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1122/2236] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:38:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:38:48.767878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.778947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013ccac5-c013-40cd-aea8-1eaceb9dbbac None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.784300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.784922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.785201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.785468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.787066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.788127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.824787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1115/2237] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:38:48 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:48.829365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.850121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb02b6-d550-48fc-b2e0-3f6d186299bb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.920390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.921216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.921499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.921771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.922454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.924814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.950528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1123/2238] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:38:48 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:38:48.954968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:48.967487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e622ca3-9137-4997-87ec-355515f88d74 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:48.972815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:48.973455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:48.973730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:48.973999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:48.974659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:48.976429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:48.992309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1116/2239] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:38:48.993948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:49.006228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa95291-ea12-4b1d-9b84-1709d605c86b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:49.011277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:49.011909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:49.012187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:49.012492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:49.013144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:49.014780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fdddc-68d2-4eb0-a6db-4325207e4f75 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:49.031272 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1124/2240] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:38:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:38:49.032892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:49.044383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78bfe26-566c-460b-b536-a82c24109652 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:49.050935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:49.052184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:49.052467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:49.052737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] PATH_INFO: `/v3/projects/ccfe0e0a4e0d4f50887c5670bf69c3c2/users/febee03afdc640b78a5699e57f64316b/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:49.070645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfe0e0a4e0d4f50887c5670bf69c3c2, user_id=febee03afdc640b78a5699e57f64316b, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:49.073710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:49.089227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1117/2241] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:38:49 2022] PUT /identity/v3/projects/ccfe0e0a4e0d4f50887c5670bf69c3c2/users/febee03afdc640b78a5699e57f64316b/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:49.107226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:49.107621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:49.107934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:49.140098 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] MFA Rules not processed for user `febee03afdc640b78a5699e57f64316b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:49.141409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:49.175654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:49.215284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1125/2242] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:49.226907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:49.228634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:49.294368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6555e6-9f0b-4153-9ef8-1fcc669d68e2 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:49.298279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:49.298930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:49.299235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:49.299495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:49.309969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:49.311476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181ab276-f86a-4277-8919-5a1deac058bd service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:49.324514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1118/2243] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:38:59.211509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=12 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:59.317452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:38:59.372220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:59.443037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e191c3-9931-48e4-98ab-fa0b23a09935 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:59.448816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:59.449496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:59.449783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:59.450050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] PATH_INFO: `/v3/users/febee03afdc640b78a5699e57f64316b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:59.451286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=febee03afdc640b78a5699e57f64316b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:59.453294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:59.517244 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febee03afdc640b78a5699e57f64316b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:59.526328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febee03afdc640b78a5699e57f64316b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:59.527116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febee03afdc640b78a5699e57f64316b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:59.527808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febee03afdc640b78a5699e57f64316b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:59.536812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febee03afdc640b78a5699e57f64316b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:38:59.556265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1126/2244] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:38:59 2022] DELETE /identity/v3/users/febee03afdc640b78a5699e57f64316b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:38:59.561081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db902290-33fb-4660-a452-d0775114d869 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:59.561372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db902290-33fb-4660-a452-d0775114d869 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:59.561851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db902290-33fb-4660-a452-d0775114d869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:59.588058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-db902290-33fb-4660-a452-d0775114d869 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:38:59.589315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-db902290-33fb-4660-a452-d0775114d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:59.661816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1119/2245] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:38:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:38:59.671928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:38:59.673466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:38:59.746374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c93bf72-0aa5-4ec1-a799-528051749e97 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:38:59.755911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:38:59.755911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:38:59.755911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:38:59.755911 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:38:59.762194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:38:59.763948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:38:59.774764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1127/2246] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:38:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:39:00.070922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db902290-33fb-4660-a452-d0775114d869 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:39:00.072390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-db902290-33fb-4660-a452-d0775114d869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:00.087533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db902290-33fb-4660-a452-d0775114d869 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:39:00.092410 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:39:00.093092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:00.093370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:00.093642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] PATH_INFO: `/v3/projects/ccfe0e0a4e0d4f50887c5670bf69c3c2` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:00.094810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfe0e0a4e0d4f50887c5670bf69c3c2)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:39:00.096962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:39:00.148214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfe0e0a4e0d4f50887c5670bf69c3c2', 'request_id': 'req-0462af1d-41bc-4ef8-8430-9e40c02b01d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:00.154785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfe0e0a4e0d4f50887c5670bf69c3c2', 'request_id': 'req-0462af1d-41bc-4ef8-8430-9e40c02b01d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:00.163572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfe0e0a4e0d4f50887c5670bf69c3c2', 'request_id': 'req-0462af1d-41bc-4ef8-8430-9e40c02b01d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:00.177253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] The token cache is being invalidate because project ccfe0e0a4e0d4f50887c5670bf69c3c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:39:00.177489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:00.178080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-0462af1d-41bc-4ef8-8430-9e40c02b01d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:00.179822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1120/2247] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:39:00 2022] DELETE /identity/v3/projects/ccfe0e0a4e0d4f50887c5670bf69c3c2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:39:04.453638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:39:04.455343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:04.469154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedb779e-3b1c-4520-b061-77f1d1809055 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:39:04.488875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:39:04.531517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:39:04.532163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:04.532436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:04.532709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] PATH_INFO: `/v3/users/fcb6b58e1f804ca7aa86cfe87fc8c3c7` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:04.533865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcb6b58e1f804ca7aa86cfe87fc8c3c7)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:39:04.535883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:39:04.594819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcb6b58e1f804ca7aa86cfe87fc8c3c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:04.602270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcb6b58e1f804ca7aa86cfe87fc8c3c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:04.603026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcb6b58e1f804ca7aa86cfe87fc8c3c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:04.603769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcb6b58e1f804ca7aa86cfe87fc8c3c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:04.613038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcb6b58e1f804ca7aa86cfe87fc8c3c7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:04.635651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1128/2248] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:39:04 2022] DELETE /identity/v3/users/fcb6b58e1f804ca7aa86cfe87fc8c3c7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:39:04.640852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:04.641394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:04.641394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:04.662352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:39:04.663623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:04.728618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1121/2249] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:39:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:39:04.739455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:39:04.741128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:04.802636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866d7f7-407b-47c5-92a8-b03ef03ef33f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:39:04.806711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:39:04.807472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:04.807778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:04.808133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:04.819015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:39:04.820517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90ef56-84ba-4ece-941b-6a4b6bfc38ea service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:39:04.831274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1129/2250] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:39:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:39:05.116514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:39:05.118063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:05.130798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba016bb-443b-46b0-875e-9398568eb45b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:39:05.135686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:39:05.136369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:05.136672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:05.136903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] PATH_INFO: `/v3/projects/25493e78c6c348ceb4f45faf2c4f9dd0` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:05.138000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25493e78c6c348ceb4f45faf2c4f9dd0)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:39:05.139956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:39:05.162586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:39:05.190180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25493e78c6c348ceb4f45faf2c4f9dd0', 'request_id': 'req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:05.195657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25493e78c6c348ceb4f45faf2c4f9dd0', 'request_id': 'req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:05.205616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25493e78c6c348ceb4f45faf2c4f9dd0', 'request_id': 'req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:05.217238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] The token cache is being invalidate because project 25493e78c6c348ceb4f45faf2c4f9dd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:39:05.217527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:05.218301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc4d76a3-8cac-4bd3-a759-56d411e46bb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:39:05.220037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1122/2251] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:39:05 2022] DELETE /identity/v3/projects/25493e78c6c348ceb4f45faf2c4f9dd0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:39:14.826561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:39:15.213934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:39:36.663018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:36.663645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:36.664107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:36.696716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] MFA Rules not processed for user `fce6a7d7dbc94adeb0b03bae34e335a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:39:36.697972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:36.758753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1130/2252] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:39:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:39:46.749960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=25 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:39:57.942999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:57.943521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:57.943615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:57.964714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:39:57.965955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:58.027960 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1123/2253] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:39:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:39:58.084032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:39:58.085806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:39:58.169725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23a79d-d641-4e89-9046-fd575918a77d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:39:58.175844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:39:58.176341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:39:58.176628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:39:58.176942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:39:58.185980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:39:58.187650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:39:58.196154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1131/2254] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:39:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:40:08.020508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=25 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:08.192685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=32 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:10.592214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:10.593803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:10.653177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b55d75-639e-4667-bab1-37a4d339fba4 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:10.657207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:10.657813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:10.658087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:10.658352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:10.660236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:10.738076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:10.739569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f513371-6de5-4476-83b7-7e36cb589a5a service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:10.750693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1124/2255] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:11.956353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:11.971922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0701ab5-b448-40c6-9c8c-81df267041e4 service swift] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:11.976750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:11.977349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:11.977621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:11.977886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:11.980135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:11.995110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:11.996547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2859a-9d09-434e-96e9-178d5ce6125f service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:12.007645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1132/2256] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:12.768478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:12.769083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:12.769273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:12.770792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1125/2257] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:40:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:40:12.777756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3fe0b-0e1d-4bd6-b1c3-ab1e0685aa9c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:12.778044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3fe0b-0e1d-4bd6-b1c3-ab1e0685aa9c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:12.778316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3fe0b-0e1d-4bd6-b1c3-ab1e0685aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:12.805473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-bde3fe0b-0e1d-4bd6-b1c3-ab1e0685aa9c None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:40:12.807055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bde3fe0b-0e1d-4bd6-b1c3-ab1e0685aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:12.818556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1133/2258] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:40:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:40:12.830937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:12.832928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:12.894537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a161b93-b132-4755-a378-d58b798cbe88 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:12.898175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:12.898791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:12.899088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:12.899395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:12.911874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:12.913332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b697d8-0d33-43be-b3fa-cc0ef4cb09c8 service glance] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:12.924067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1126/2259] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:13.003284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7f99f-db4e-40f7-8557-cbe06335440f None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.003843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7f99f-db4e-40f7-8557-cbe06335440f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.003843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b7f99f-db4e-40f7-8557-cbe06335440f None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.005131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1134/2260] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 08 13:40:13.010522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd21e9-0ae0-4859-9f14-fea2f1c0ad41 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.010787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd21e9-0ae0-4859-9f14-fea2f1c0ad41 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.011119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd21e9-0ae0-4859-9f14-fea2f1c0ad41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.036324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d2dd21e9-0ae0-4859-9f14-fea2f1c0ad41 None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:40:13.037530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d2dd21e9-0ae0-4859-9f14-fea2f1c0ad41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:13.045193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1127/2261] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:40:13.057504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7f99f-db4e-40f7-8557-cbe06335440f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:13.069906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b7f99f-db4e-40f7-8557-cbe06335440f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:13.073854 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:13.074452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.074719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.074984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.082235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:13.083729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585347c1-e0c1-44a9-a482-26804c38716d service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:13.092370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1135/2262] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:13.177845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1b8f-d2e7-40db-8314-3bf6d4552dfa None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.178133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1b8f-d2e7-40db-8314-3bf6d4552dfa None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.178432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb1b8f-d2e7-40db-8314-3bf6d4552dfa None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.180104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1128/2263] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:40:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:40:13.185619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ba8e-f00c-4c02-ada4-3e21c27aa1b9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.185904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ba8e-f00c-4c02-ada4-3e21c27aa1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.186176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571ba8e-f00c-4c02-ada4-3e21c27aa1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.207748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a571ba8e-f00c-4c02-ada4-3e21c27aa1b9 None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:40:13.208966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a571ba8e-f00c-4c02-ada4-3e21c27aa1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:13.219465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1136/2264] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:40:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:40:13.230140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1b8f-d2e7-40db-8314-3bf6d4552dfa None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:13.243044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb1b8f-d2e7-40db-8314-3bf6d4552dfa None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:13.248071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:13.248687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:13.248961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:13.249225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:13.261261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:13.262638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:13.274209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1129/2265] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:40:16.233216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475507-e39c-4405-841c-2b39c908831d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:16.233717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475507-e39c-4405-841c-2b39c908831d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:16.233717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a475507-e39c-4405-841c-2b39c908831d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:16.254839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9a475507-e39c-4405-841c-2b39c908831d None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:40:16.256097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9a475507-e39c-4405-841c-2b39c908831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:16.264206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1137/2266] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:40:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:40:16.276845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:16.289462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6b766-0965-4e05-b92b-a7b7a9428406 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:16.296114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:16.296723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:16.297009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:16.297271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:16.306393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:16.308008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:16.316116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1130/2267] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:40:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 13:40:26.249030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=21 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:26.312827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:31.484482 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a475507-e39c-4405-841c-2b39c908831d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:31.486038 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9a475507-e39c-4405-841c-2b39c908831d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:31.550511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a475507-e39c-4405-841c-2b39c908831d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:31.554884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:31.555533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:31.555795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:31.556070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:31.558119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:31.628765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:31.630214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527be73-1ae2-467e-9d07-7f48f6f4a9c5 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:31.640136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1138/2268] 10.222.0.33 () {60 vars in 1396 bytes} [Mon Aug 8 13:40:31 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3270 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:40.700439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:40.702014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:40.717399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218beaa-826a-4a6c-943e-dac11a8daeb9 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:40.721870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:40.722503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:40.722779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:40.723054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:40.725320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:40.737377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:40.738853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:40.748512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1131/2269] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:40:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:40:41.635309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=48 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:45.674777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f01ae7-c7da-4fa0-a2f2-5c751952a6a7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:45.675491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f01ae7-c7da-4fa0-a2f2-5c751952a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:45.675491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f01ae7-c7da-4fa0-a2f2-5c751952a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:45.695700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-81f01ae7-c7da-4fa0-a2f2-5c751952a6a7 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:40:45.696921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-81f01ae7-c7da-4fa0-a2f2-5c751952a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:40:45.712835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1139/2270] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:40:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:40:45.724429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:40:45.737416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cae25d3-4757-4afe-a8c8-fe5aa11ed7a9 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:40:45.743315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:40:45.744018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:40:45.744349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:40:45.744655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:40:45.754158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:40:45.755784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:40:45.764643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1132/2271] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:40:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:40:55.689372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:40:55.760865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=12 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:02.241451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:02.242022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:02.242022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:02.263176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] MFA Rules not processed for user `518d6948777944fd8214dc6d778a9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:41:02.264531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:02.273298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1140/2272] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:41:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:41:07.351777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:07.353262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:07.370374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d634a36-5b02-4237-b167-a2895d5d5dd5 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:07.381112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:07.381112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:07.381112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:07.381112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:07.383042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:07.396049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:07.397680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:07.409122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1133/2273] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:10.916254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:10.934164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf138e44-af6a-4e04-a56d-65c05ab31727 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:10.940771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:10.941406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:10.941681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:10.941948 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:10.943899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:11.016221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:11.016667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:11.027888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1141/2274] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:17.405008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:20.073617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:20.085210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab35388-c473-4c56-a43e-3a205e1b44be service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:20.092423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:20.093034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:20.093310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:20.093576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:20.100780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:20.113460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:20.115102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68292f83-ea3e-473a-bf9f-e2a893ea33e9 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:20.124928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1134/2275] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:21.023291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=31 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:23.536946 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:23.538429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:23.553020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c09f76-01a4-49ab-b27d-ba1b8b88e427 service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:23.560091 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:23.561086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:23.561298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:23.561564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:23.563514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:23.577196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:23.578870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:23.590896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1142/2276] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:41:30.120268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:33.586307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:35.668473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:35.669140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:35.669140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:35.694545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] MFA Rules not processed for user `fce6a7d7dbc94adeb0b03bae34e335a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:41:35.695931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:35.704569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1135/2277] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:41:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:41:36.028172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:36.043712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59816d3-830e-46c5-bf6b-b46fda7fdcdd service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:36.048703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:36.049313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:36.049593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:36.049874 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:36.052289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:36.063809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:36.065241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:36.074397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1143/2278] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:36.408731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:36.410250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:36.486203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a626da-bf27-4bbe-8c81-520a3daac54b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:36.491843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:36.492476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:36.492783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:36.493056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:36.494885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:36.509401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:36.511039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:36.522677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1136/2279] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:46.070139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:46.517631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=35 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:41:46.930703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:46.945735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e330a-e0f2-4c84-b969-e4ff0d8ffa64 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:46.952713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:46.953336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:46.953825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:46.954095 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:46.956049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:46.970220 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:46.972462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ca587-7ecc-4270-9383-0a198381a421 service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:46.986080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1144/2280] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:53.784947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:41:53.801583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0601f58-e671-476e-8068-867afcd658e4 service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:41:53.808837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:41:53.809448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:41:53.809724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:41:53.809989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:41:53.812081 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:41:53.876200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:41:53.877808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:41:53.887524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1137/2281] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:41:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:41:56.980026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:02.556478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:02.557165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:02.557165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:02.580461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] MFA Rules not processed for user `c29ce05ebc6e4672bb9557f751a48557`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:02.581685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:02.589113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1145/2282] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:03.882806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=28 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:12.574936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:27.886501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:27.896627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b1c00-dd92-4bd4-8e00-85bf20994b3a service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:27.900939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:27.901568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:27.901847 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:27.902132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:27.904166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:27.972068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:27.973533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:27.984559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1138/2283] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:31.402554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:31.403723 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:31.426726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874835cd-c398-4cc6-8f72-281104b749fb None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:31.435032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:31.435766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:31.436094 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:31.436322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:31.438474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:31.451464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:31.453227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:31.465676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1146/2284] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:31.470837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:31.472730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:31.490909 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea49f8-9af5-42fb-9f20-33d4b5f4c501 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:31.495203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:31.495908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:31.496183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:31.496625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] PATH_INFO: `/v3/users/518d6948777944fd8214dc6d778a9a02` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:31.497736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] RBAC: Authorizing `identity:delete_user(user_id=518d6948777944fd8214dc6d778a9a02)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:31.499769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:31.552053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '518d6948777944fd8214dc6d778a9a02'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.559524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '518d6948777944fd8214dc6d778a9a02'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.573300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '518d6948777944fd8214dc6d778a9a02'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.574064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '518d6948777944fd8214dc6d778a9a02'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.574899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-32cbd6f6-2709-4541-b0c2-997cd2f87786 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '518d6948777944fd8214dc6d778a9a02'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.594312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1139/2285] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:42:31 2022] DELETE /identity/v3/users/518d6948777944fd8214dc6d778a9a02 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:31.784198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:31.797034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0757eb7-c5a1-4341-ad75-25dd6c27d129 service placement] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:31.858576 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:31.859200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:31.859477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:31.859748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] PATH_INFO: `/v3/projects/f90db8c721374e0fb3a037661b3d9fe0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:31.860893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90db8c721374e0fb3a037661b3d9fe0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:31.862820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:31.923745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90db8c721374e0fb3a037661b3d9fe0', 'request_id': 'req-c2d44904-0fdf-4570-ab6a-1f409f479e92'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.929803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90db8c721374e0fb3a037661b3d9fe0', 'request_id': 'req-c2d44904-0fdf-4570-ab6a-1f409f479e92'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.939420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90db8c721374e0fb3a037661b3d9fe0', 'request_id': 'req-c2d44904-0fdf-4570-ab6a-1f409f479e92'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.951333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] The token cache is being invalidate because project f90db8c721374e0fb3a037661b3d9fe0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:31.951624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.952256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c2d44904-0fdf-4570-ab6a-1f409f479e92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:31.953858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1147/2286] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:31 2022] DELETE /identity/v3/projects/f90db8c721374e0fb3a037661b3d9fe0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:31.958848 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:31.959179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:31.959484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:31.981604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:31.982830 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:32.051482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1140/2287] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:32.059325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.059610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.059883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.080607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:32.081862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:32.090776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1148/2288] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:32.095279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.106366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5439e5a-3e46-4042-a068-aebb8c613d3f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.112567 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.113117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.113396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.113664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.114330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:42:32.114631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.116428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.118328 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:32.136595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1141/2289] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:42:32.138125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.149018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb670b-60a5-4146-8c14-a2e1bdc8b460 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.154743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.155413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.155776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.156025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.156845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.158826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.193206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1149/2290] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:32.198337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.211387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fbc95-ccc8-4b0b-8073-7ad53f17033f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.272398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.273145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.273298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.273544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.274180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.276222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.299496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1142/2291] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/users => generated 278 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:32.304728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.314513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e216841-fdca-40c6-89cd-3fdfd4ccd1b2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.319390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.319999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.320280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.320554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.321250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.322838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.332933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1150/2292] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:42:32.337240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.348116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98267a06-b88a-4be9-9243-e3bbf2b1a93c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.353978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.354603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.354869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.355168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.355809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.357425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ef2942-37e8-4259-b603-95a763f474f7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.369024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1143/2293] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:32.373353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.383824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a5d57-95b6-4e40-b108-372c12e0733a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.389131 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.389800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.390104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.390375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] PATH_INFO: `/v3/projects/724d822c156c439b99257bae96b133f5/users/08e2789c02f14ae182dd8a9b1d3af53c/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.405301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=724d822c156c439b99257bae96b133f5, user_id=08e2789c02f14ae182dd8a9b1d3af53c, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.408107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.427442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1151/2294] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:32 2022] PUT /identity/v3/projects/724d822c156c439b99257bae96b133f5/users/08e2789c02f14ae182dd8a9b1d3af53c/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:32.442139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.442513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.442850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.474426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] MFA Rules not processed for user `08e2789c02f14ae182dd8a9b1d3af53c`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:32.476091 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:32.543414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1144/2295] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:32.548879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.562442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:32.622085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995b69d-1c4b-41fe-b356-bdb87e223034 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.627228 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.627824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.628102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.628373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.629002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.630914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.660545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1152/2296] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/projects => generated 355 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:32.664925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.677164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a791ad-efdd-4a44-865a-b45ee4bee5dd None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.739407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.740055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.740306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.740575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.741206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.743160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.752077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:32.771431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1145/2297] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:32 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:32.775994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.788050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e45517-b516-41d9-9cf1-b8c3d7254577 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.793297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.793902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.794180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.794434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.795103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.796842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.808694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1153/2298] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:32.814056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.828219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469db7c6-0150-4804-9aa6-66fbc6b14b4d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.833561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.834247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.834522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.834791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] PATH_INFO: `/v3/projects/f745bee1d6af49fe93e6771781358525/users/3c56165573ce4a479fa869da99516b94/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.851931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f745bee1d6af49fe93e6771781358525, user_id=3c56165573ce4a479fa869da99516b94, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.854867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.871350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1146/2299] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:32 2022] PUT /identity/v3/projects/f745bee1d6af49fe93e6771781358525/users/3c56165573ce4a479fa869da99516b94/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:42:32.875884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.888971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb12e2-057e-40f5-9793-9b9f3f2d7774 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.958096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.958705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.958975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.959277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.959919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:32.961529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65312544-7239-4614-980e-559bf1259fd1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:32.971984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1154/2300] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:32 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:32.976797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:32.988331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d6b8d0-b44a-41ac-8451-b18eff89c59f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:32.994303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:32.994992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:32.995312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:32.995587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] PATH_INFO: `/v3/projects/f745bee1d6af49fe93e6771781358525/users/3c56165573ce4a479fa869da99516b94/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:32.997606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f745bee1d6af49fe93e6771781358525, user_id=3c56165573ce4a479fa869da99516b94, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:33.000466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:33.013362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1147/2301] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:32 2022] PUT /identity/v3/projects/f745bee1d6af49fe93e6771781358525/users/3c56165573ce4a479fa869da99516b94/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:33.024560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:33.025070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:33.025449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:33.058679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] MFA Rules not processed for user `3c56165573ce4a479fa869da99516b94`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:33.059992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:33.146417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1155/2302] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:33.159451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:33.161208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:33.224692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb86232-99d3-45c4-b802-a43173f412b6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:33.228715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:33.229367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:33.229641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:33.229906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:33.240168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:33.241618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a7842-5475-440a-9af9-096b7fec7cbf service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:33.252007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1148/2303] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:33.358709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:33.369959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded99112-eddf-4d09-b554-d66c91f2f817 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:33.373941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:33.374697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:33.375104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:33.375466 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:33.385659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:33.459317 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:33.460825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:33.470936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1156/2304] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:39.035670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:39.036177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:39.036261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:39.055256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] MFA Rules not processed for user `651659e941044454985b2f866b791b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:39.056938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:39.121652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1149/2305] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:39.481384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:39.483021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:39.544951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27d1d9-0525-4f93-b082-368cf0964b6f service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:39.549574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:39.550179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:39.550456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:39.550725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:39.552705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:39.563306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:39.564787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3beddb56-e1e9-42e3-b6ee-45ff2b97f30e service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:39.573768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1157/2306] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:49.069207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:49.137690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b23a5c-5499-4c08-b5d7-7d77fdf41b1d None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:49.142810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:49.143509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:49.143772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:49.144043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] PATH_INFO: `/v3/users/08e2789c02f14ae182dd8a9b1d3af53c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:49.145187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08e2789c02f14ae182dd8a9b1d3af53c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:49.147288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:49.149989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:49.198130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08e2789c02f14ae182dd8a9b1d3af53c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.207563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08e2789c02f14ae182dd8a9b1d3af53c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.217840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08e2789c02f14ae182dd8a9b1d3af53c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.218822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08e2789c02f14ae182dd8a9b1d3af53c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.219744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08e2789c02f14ae182dd8a9b1d3af53c'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.240940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1150/2307] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:42:49 2022] DELETE /identity/v3/users/08e2789c02f14ae182dd8a9b1d3af53c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:49.245916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:49.246475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:49.246475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:49.267690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:49.268990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:49.335166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1158/2308] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:49.345614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:49.347258 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:49.434518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43729556-15dc-4113-9109-62a64049df07 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:49.441057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:49.441833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:49.442169 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:49.442445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:49.453181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:49.455003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:49.466870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1151/2309] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:42:49.821361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:49.823070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:49.838382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35fb8bc-0427-4609-a9b1-b76f924ac6db None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:49.845943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:49.846504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:49.846770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:49.847129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] PATH_INFO: `/v3/projects/724d822c156c439b99257bae96b133f5` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:49.848370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=724d822c156c439b99257bae96b133f5)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:49.850633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:49.898508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:49.902173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '724d822c156c439b99257bae96b133f5', 'request_id': 'req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.908649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '724d822c156c439b99257bae96b133f5', 'request_id': 'req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.915795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '724d822c156c439b99257bae96b133f5', 'request_id': 'req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.927183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] The token cache is being invalidate because project 724d822c156c439b99257bae96b133f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:49.927472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.928112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:49.929619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1159/2310] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:49 2022] DELETE /identity/v3/projects/724d822c156c439b99257bae96b133f5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:49.933973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:49.944238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2631cc99-e8e8-446f-8279-b4a36d022623 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.006760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.007489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.007845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.008386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] PATH_INFO: `/v3/users/3c56165573ce4a479fa869da99516b94` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.010057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c56165573ce4a479fa869da99516b94)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.012371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.046740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:50.083657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c56165573ce4a479fa869da99516b94'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.093008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c56165573ce4a479fa869da99516b94'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.102333 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c56165573ce4a479fa869da99516b94'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.102976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c56165573ce4a479fa869da99516b94'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.103677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2bcc4b3b-c770-4219-aa45-aded946a9334 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c56165573ce4a479fa869da99516b94'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.124060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1152/2311] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:42:49 2022] DELETE /identity/v3/users/3c56165573ce4a479fa869da99516b94 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:50.428534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.430010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:50.444758 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ed62b-85ca-41a5-98ff-01bf4c4c62f0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.507423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.508449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.508812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.509084 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] PATH_INFO: `/v3/projects/f745bee1d6af49fe93e6771781358525` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.510503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f745bee1d6af49fe93e6771781358525)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.513570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.568878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f745bee1d6af49fe93e6771781358525', 'request_id': 'req-cc23d499-a178-4aa9-8c91-98d518db0380'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.574282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f745bee1d6af49fe93e6771781358525', 'request_id': 'req-cc23d499-a178-4aa9-8c91-98d518db0380'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.582687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f745bee1d6af49fe93e6771781358525', 'request_id': 'req-cc23d499-a178-4aa9-8c91-98d518db0380'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.594508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] The token cache is being invalidate because project f745bee1d6af49fe93e6771781358525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:50.594796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.595455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cc23d499-a178-4aa9-8c91-98d518db0380 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:50.597026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1160/2312] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:50 2022] DELETE /identity/v3/projects/f745bee1d6af49fe93e6771781358525 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:50.602611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.602904 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.603205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.624487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:50.625720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:50.696286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1153/2313] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:50.705165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.705447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.705718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.727444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:50.728681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:50.738085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1161/2314] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:50.742433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.752555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45538cf-b9b6-4f25-b6f4-a6b030651150 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.757306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.757917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.758191 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.758455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.759104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:42:50.759430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.761078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.772853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1154/2315] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:42:50.777419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.787597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e603e-7adc-4e5e-af3b-368c04f96d39 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.794563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.795219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.795525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.795903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.796599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.798636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.831461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1162/2316] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:50 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:50.836048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.846728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c234cc-2008-4774-8da4-c6343729abb5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.916481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.917210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.917556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.917900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.918616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.920796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.944263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1155/2317] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:50 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:42:50.948939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.960273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683129cb-83ca-441f-9fb6-4d0f05b8f13e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:50.965400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:50.966010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:50.966285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:50.966551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:50.967238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:50.969004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:50.980282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1163/2318] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:50.984768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:50.995451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cc0a2c-fe5d-48c7-80db-3f3533c44351 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.003047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.003691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.003963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.004238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.005699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.006576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689541ff-a743-4d03-a908-805bb3a38fab admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.020401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1156/2319] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:51.025213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.040503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60611cb3-5b6f-4fa0-8a3e-2877a668829a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.045507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.046198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.046471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.046742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] PATH_INFO: `/v3/projects/5f500a3504774c85abbc312c7133a7fb/users/cf42946c652844cd949914f0c3b44634/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.064401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f500a3504774c85abbc312c7133a7fb, user_id=cf42946c652844cd949914f0c3b44634, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.067148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.076833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:51.083207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1164/2320] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:51 2022] PUT /identity/v3/projects/5f500a3504774c85abbc312c7133a7fb/users/cf42946c652844cd949914f0c3b44634/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:51.094612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.095209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.095209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.119801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:51.130582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] MFA Rules not processed for user `cf42946c652844cd949914f0c3b44634`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:51.132512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:51.203292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1157/2321] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:51.207860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.276315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec560a-af8c-4cbb-be06-3d4dd811abae admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.281744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.282379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.282652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.282902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.283601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.286268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.319430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1165/2322] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:51 2022] POST /identity/v3/projects => generated 381 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:51.320787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.337533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4935f61f-4c3b-4dc2-9e32-96963e98be25 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.404812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.405480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.405760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.406030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.406693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.408711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.431229 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1158/2323] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:51 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:51.435375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.446356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695dcba2-f2d1-4f58-8b56-a3aab4de8846 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.451346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.452036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.452528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.452776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.453643 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.455447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.467900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1166/2324] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:51.472763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.484943 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797f0123-e4ab-4508-9da1-9d659f403582 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.495861 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.496549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.496836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.497111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] PATH_INFO: `/v3/projects/80d8242525ee4bf19276f7503ed26ed2/users/71c025c012e94b18b317b433247f64a0/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.516214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d8242525ee4bf19276f7503ed26ed2, user_id=71c025c012e94b18b317b433247f64a0, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.518953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.535263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1159/2325] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:51 2022] PUT /identity/v3/projects/80d8242525ee4bf19276f7503ed26ed2/users/71c025c012e94b18b317b433247f64a0/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:51.539843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.550593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb0d21-a038-4b61-976b-5ff035c763a0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.618906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.620001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.620548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.621023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.621859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.624484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd3338c-e717-43dd-86ee-aca4d13c6d9d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.639658 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1167/2326] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:51.644238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.655192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbbb23-f3c7-4b96-a5a2-b91d8fe63b42 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:51.660921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:51.661932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.662374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.662776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] PATH_INFO: `/v3/projects/80d8242525ee4bf19276f7503ed26ed2/users/71c025c012e94b18b317b433247f64a0/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.665624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d8242525ee4bf19276f7503ed26ed2, user_id=71c025c012e94b18b317b433247f64a0, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:51.670028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:51.686041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1160/2327] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:51 2022] PUT /identity/v3/projects/80d8242525ee4bf19276f7503ed26ed2/users/71c025c012e94b18b317b433247f64a0/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:51.699116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:51.699467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:51.699686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:51.730878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] MFA Rules not processed for user `71c025c012e94b18b317b433247f64a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:51.732050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:51.800628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:51.812014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1168/2328] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:51.825015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:51.827024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:52.037083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db4ad2-a847-4313-8b58-6705c6b7ef8d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:52.041049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:52.041681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:52.041958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:52.042228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:52.049437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:52.059786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:52.061379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:52.073789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1161/2329] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:52.442320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:52.452897 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed3ffe-a972-4ccb-a289-75f769876681 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:52.458248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c025c012e94b18b317b433247f64a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80d8242525ee4bf19276f7503ed26ed2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:52.458847 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:52.459157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:52.459438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:52.460113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:42:52.460425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:52.462080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:52.484416 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1169/2330] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:42:52.489558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:52.500521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1e7bf-2bb7-4e3a-8361-1c47827411e9 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:52.505442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c025c012e94b18b317b433247f64a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80d8242525ee4bf19276f7503ed26ed2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:52.506079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:52.506361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:52.506641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:52.507323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:52.509283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:52.540747 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1162/2331] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:52 2022] POST /identity/v3/projects => generated 353 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:42:52.545421 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:52.560057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63913165-21fb-4490-9614-34b5ffdb4c0c tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:52.657371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c025c012e94b18b317b433247f64a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80d8242525ee4bf19276f7503ed26ed2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:52.658427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:52.658932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:52.659447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:52.660668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:42:52.661226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:52.664475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:52.688064 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1170/2332] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:42:52.730002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:52.741475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4661fb-6706-46ca-bc4a-7509f83dafa3 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:52.747620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71c025c012e94b18b317b433247f64a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80d8242525ee4bf19276f7503ed26ed2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:52.748302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:52.748582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:52.748942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] PATH_INFO: `/v3/projects/128e35903dff474984618930df38be97` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:52.750357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorizing `identity:delete_project(project_id=128e35903dff474984618930df38be97)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:52.752503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:52.827247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '128e35903dff474984618930df38be97', 'request_id': 'req-8ecf680a-bce7-40ae-a8b1-b6088a50b220'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:52.834129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '128e35903dff474984618930df38be97', 'request_id': 'req-8ecf680a-bce7-40ae-a8b1-b6088a50b220'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:52.843556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '128e35903dff474984618930df38be97', 'request_id': 'req-8ecf680a-bce7-40ae-a8b1-b6088a50b220'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:52.859934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] The token cache is being invalidate because project 128e35903dff474984618930df38be97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:52.860771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:52.862616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8ecf680a-bce7-40ae-a8b1-b6088a50b220 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:52.864897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1163/2333] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:52 2022] DELETE /identity/v3/projects/128e35903dff474984618930df38be97 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:53.104542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:53.112109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:53.190270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb5020-fcea-4ed8-a361-2a9c9fe95991 tempest-VolumeQuotaClassesTest-2133846311 tempest-VolumeQuotaClassesTest-2133846311-project-admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:53.251456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:53.251975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:53.252257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:53.252655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] PATH_INFO: `/v3/users/cf42946c652844cd949914f0c3b44634` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:53.253773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf42946c652844cd949914f0c3b44634)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:53.256157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:53.299069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:53.321219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf42946c652844cd949914f0c3b44634'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:53.329435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf42946c652844cd949914f0c3b44634'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:53.330225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf42946c652844cd949914f0c3b44634'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:53.330877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf42946c652844cd949914f0c3b44634'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:53.339950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf42946c652844cd949914f0c3b44634'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:53.360622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1171/2334] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:42:53 2022] DELETE /identity/v3/users/cf42946c652844cd949914f0c3b44634 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:42:53.365275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:53.365579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:53.365832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:53.400750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:53.402762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:53.505860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1164/2335] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:53.515724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:53.517321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:53.593279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3c9f6-7fac-4253-b480-21a8c925aa92 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:53.598062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:53.599823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:53.600215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:53.601429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:53.611579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:53.613197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:53.626227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1172/2336] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:42:53.985668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:53.987343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:54.006183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d8051-cb1c-4673-ae92-9185996e6afd None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:54.011289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:54.012087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:54.012421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:54.012961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] PATH_INFO: `/v3/projects/5f500a3504774c85abbc312c7133a7fb` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:54.014267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f500a3504774c85abbc312c7133a7fb)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:54.016405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:54.090338 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f500a3504774c85abbc312c7133a7fb', 'request_id': 'req-a100ce86-b1a7-47f6-b197-8ae89fd311e4'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.093934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:54.101359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f500a3504774c85abbc312c7133a7fb', 'request_id': 'req-a100ce86-b1a7-47f6-b197-8ae89fd311e4'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.121407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f500a3504774c85abbc312c7133a7fb', 'request_id': 'req-a100ce86-b1a7-47f6-b197-8ae89fd311e4'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.137583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] The token cache is being invalidate because project 5f500a3504774c85abbc312c7133a7fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:54.138004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.138725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.140458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1165/2337] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:53 2022] DELETE /identity/v3/projects/5f500a3504774c85abbc312c7133a7fb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:54.144602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:54.154808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026ad89-4105-4027-aadf-79370d15db10 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:54.262112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:54.262768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:54.263045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:54.263361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] PATH_INFO: `/v3/users/71c025c012e94b18b317b433247f64a0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:54.264687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71c025c012e94b18b317b433247f64a0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:54.266751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:54.340180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71c025c012e94b18b317b433247f64a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.341888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:54.351463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71c025c012e94b18b317b433247f64a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.352805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71c025c012e94b18b317b433247f64a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.354529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71c025c012e94b18b317b433247f64a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.379278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-024b722c-f2c6-4c81-9675-21042a67a20b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71c025c012e94b18b317b433247f64a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.418924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1173/2338] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:42:54 2022] DELETE /identity/v3/users/71c025c012e94b18b317b433247f64a0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:42:54.784190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:54.785907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:54.801032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a100ce86-b1a7-47f6-b197-8ae89fd311e4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:54.906382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:54.907430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:54.907778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:54.908124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] PATH_INFO: `/v3/projects/80d8242525ee4bf19276f7503ed26ed2` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:54.909374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d8242525ee4bf19276f7503ed26ed2)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:54.911440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:54.967110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d8242525ee4bf19276f7503ed26ed2', 'request_id': 'req-513e93bc-6cf1-43b8-8712-44a3c3d6e971'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.974135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d8242525ee4bf19276f7503ed26ed2', 'request_id': 'req-513e93bc-6cf1-43b8-8712-44a3c3d6e971'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:54.989348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d8242525ee4bf19276f7503ed26ed2', 'request_id': 'req-513e93bc-6cf1-43b8-8712-44a3c3d6e971'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:55.002016 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] The token cache is being invalidate because project 80d8242525ee4bf19276f7503ed26ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:42:55.002309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:55.002975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-513e93bc-6cf1-43b8-8712-44a3c3d6e971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:42:55.004513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1166/2339] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:42:54 2022] DELETE /identity/v3/projects/80d8242525ee4bf19276f7503ed26ed2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:55.009617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.010197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.010197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.033808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:55.035237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:55.104070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1174/2340] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:55.113797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.114078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.114351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.135527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:55.136777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:55.146424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1167/2341] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:55.151434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.166173 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e3a6d-1c91-4867-9711-fc52a4460b16 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.172577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.173201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.173474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.173750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.174431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:42:55.174817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.176514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.189691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1175/2342] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:42:55.197731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.207463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed101a9-06ff-4d1c-8c81-4c861894705a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.213654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.214305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.214608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.214872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.215565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.217557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.253206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1168/2343] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:55.259479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.272210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db8c78-47af-4ca1-ae2c-ce5d59250227 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.333771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.334288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.334561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.334832 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.335497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.337430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.360606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1176/2344] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:55.365927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.384579 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8851653-cd5e-43fb-bb60-788d1e404d0f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.392181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.392818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.393085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.393349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.394010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.395677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.406748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1169/2345] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:42:55.411569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.423567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6c245-62f1-4109-87c2-ccd9d83769fc admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.431443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.432128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.432485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.432890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.433762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.436029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ec3f44-f781-42c1-ad66-e03949ea5df9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.447965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1177/2346] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:42:55.452469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.466400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47fdc6d-e7f0-4a6a-8d76-9ae87b2112ca admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.472722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.473390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.473666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.473933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] PATH_INFO: `/v3/projects/990cb7bcfeb44c268affec34822082ec/users/a3c306437e1349d7ae76fe61a6a514f0/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.492980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=990cb7bcfeb44c268affec34822082ec, user_id=a3c306437e1349d7ae76fe61a6a514f0, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.495680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.510448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1170/2347] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:55 2022] PUT /identity/v3/projects/990cb7bcfeb44c268affec34822082ec/users/a3c306437e1349d7ae76fe61a6a514f0/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:55.520954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.521247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.521590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.550965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] MFA Rules not processed for user `a3c306437e1349d7ae76fe61a6a514f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:55.552241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:55.574811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:55.625029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1178/2348] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:55.631499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.637631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:55.699415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5643901-c041-4637-9f8f-d6ad28afe390 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.704696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.705296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.705573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.705839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.706458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.708402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.738470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1171/2349] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:55.743071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.755657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5558ca49-0f21-43a6-9b70-12e829e99856 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.837690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.838318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.838680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.838858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.839579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.841526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.868071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1179/2350] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:42:55 2022] POST /identity/v3/users => generated 290 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:42:55.872362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.883146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1039123d-7b26-4d43-afcd-1ed6c19d73a6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.887992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.888604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.888876 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.889172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.889833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.891478 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.902055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1172/2351] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:42:55.906549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.916801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a65ed-2254-4163-abd7-4d140d13b61b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:55.922201 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:55.922848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:55.923176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:55.923446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] PATH_INFO: `/v3/projects/d036a7b8844843af9f1103716b19f920/users/469d0ebcf4444a3997d0ad2b983dc6d7/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:55.941612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=d036a7b8844843af9f1103716b19f920, user_id=469d0ebcf4444a3997d0ad2b983dc6d7, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:55.945893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:55.966136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1180/2352] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:55 2022] PUT /identity/v3/projects/d036a7b8844843af9f1103716b19f920/users/469d0ebcf4444a3997d0ad2b983dc6d7/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:55.971215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:55.986090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e7f714-32b4-45fc-8d39-b2a11a1b3033 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:56.048884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:56.049506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:56.049783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:56.050058 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:56.050708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:56.052362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2727ed-878e-4073-8217-4f096f9d994e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:56.063964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1173/2353] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:42:56.068661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:56.080371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9995a-0270-440b-a67a-3c00a92b3ade admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:56.085958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:56.086617 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:56.086898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:56.087211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] PATH_INFO: `/v3/projects/d036a7b8844843af9f1103716b19f920/users/469d0ebcf4444a3997d0ad2b983dc6d7/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:56.089290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d036a7b8844843af9f1103716b19f920, user_id=469d0ebcf4444a3997d0ad2b983dc6d7, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:56.091982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:56.106105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1181/2354] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:42:56 2022] PUT /identity/v3/projects/d036a7b8844843af9f1103716b19f920/users/469d0ebcf4444a3997d0ad2b983dc6d7/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:42:56.122315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:56.122616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:56.122888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:56.153362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] MFA Rules not processed for user `469d0ebcf4444a3997d0ad2b983dc6d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:42:56.154573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:56.228508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:42:56.237818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1174/2355] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:42:56 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:42:56.248023 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:56.250207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:42:56.310333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83173c6-fe09-4786-ae3f-cde9ad57e85d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:56.314193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:56.314814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:56.315113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:56.315391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:56.326316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:56.327809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:56.337902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1182/2356] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:42:56.440084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:42:56.450087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45bf37-9178-4dd7-80ac-baf7d8b82d5a None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:42:56.454485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:42:56.455099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:42:56.455387 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:42:56.455658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:42:56.521074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:42:56.522516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd2acfe-157c-449a-97c8-994f67cd0f8e service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:42:56.532116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1175/2357] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:42:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:01.576417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:01.594134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:01.661006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9229add9-f700-4d63-b611-aab18fe80fe7 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:01.666521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:01.667178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:01.667458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:01.667725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] PATH_INFO: `/v3/users/a3c306437e1349d7ae76fe61a6a514f0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:01.669014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3c306437e1349d7ae76fe61a6a514f0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:01.670945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:01.719501 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3c306437e1349d7ae76fe61a6a514f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:01.726990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3c306437e1349d7ae76fe61a6a514f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:01.727700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3c306437e1349d7ae76fe61a6a514f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:01.728336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3c306437e1349d7ae76fe61a6a514f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:01.738344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3c306437e1349d7ae76fe61a6a514f0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:01.757436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1183/2358] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:01 2022] DELETE /identity/v3/users/a3c306437e1349d7ae76fe61a6a514f0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:01.762481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:01.763028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:01.763154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:01.784311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:01.785518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:01.850244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1176/2359] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:43:01.860318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:01.861975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:01.926158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b17805-2661-43f8-a14d-e46eba0e8f5a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:01.930171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:01.930797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:01.931102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:01.931371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:01.941670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:01.943125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:01.953662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1184/2360] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:43:02.253382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:02.255033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:02.267586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8fd7ec-8bff-4ce1-b2ed-87ac2de3e511 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:02.273539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:02.274175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:02.274431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:02.274694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] PATH_INFO: `/v3/projects/990cb7bcfeb44c268affec34822082ec` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:02.275961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=990cb7bcfeb44c268affec34822082ec)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:02.277897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:02.329026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '990cb7bcfeb44c268affec34822082ec', 'request_id': 'req-092c1ca2-86f3-4b96-baec-666403b93aa3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.335111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '990cb7bcfeb44c268affec34822082ec', 'request_id': 'req-092c1ca2-86f3-4b96-baec-666403b93aa3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.343811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '990cb7bcfeb44c268affec34822082ec', 'request_id': 'req-092c1ca2-86f3-4b96-baec-666403b93aa3'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.356223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] The token cache is being invalidate because project 990cb7bcfeb44c268affec34822082ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:02.356512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.357224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.358810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1177/2361] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:02 2022] DELETE /identity/v3/projects/990cb7bcfeb44c268affec34822082ec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:02.363424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:02.373358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46189555-fadb-4c0a-9f17-a06b75d6facc service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:02.412475 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:02.437926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:02.438481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:02.438760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:02.439034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] PATH_INFO: `/v3/users/469d0ebcf4444a3997d0ad2b983dc6d7` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:02.440281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] RBAC: Authorizing `identity:delete_user(user_id=469d0ebcf4444a3997d0ad2b983dc6d7)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:02.442223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:02.503516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469d0ebcf4444a3997d0ad2b983dc6d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.510812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469d0ebcf4444a3997d0ad2b983dc6d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.511518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469d0ebcf4444a3997d0ad2b983dc6d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.512189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469d0ebcf4444a3997d0ad2b983dc6d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.520488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f84e529a-f175-46ce-b1e9-540ca9b0d16b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469d0ebcf4444a3997d0ad2b983dc6d7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.543934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1185/2362] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:02 2022] DELETE /identity/v3/users/469d0ebcf4444a3997d0ad2b983dc6d7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:02.832155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:02.833650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:02.847313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092c1ca2-86f3-4b96-baec-666403b93aa3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:02.870126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:02.909926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:02.910562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:02.910839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:02.911137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] PATH_INFO: `/v3/projects/d036a7b8844843af9f1103716b19f920` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:02.912261 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d036a7b8844843af9f1103716b19f920)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:02.914182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:02.973126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd036a7b8844843af9f1103716b19f920', 'request_id': 'req-ad3cad19-dac3-4079-a20a-97b0d4711024'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.979564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd036a7b8844843af9f1103716b19f920', 'request_id': 'req-ad3cad19-dac3-4079-a20a-97b0d4711024'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:02.989477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd036a7b8844843af9f1103716b19f920', 'request_id': 'req-ad3cad19-dac3-4079-a20a-97b0d4711024'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.000937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] The token cache is being invalidate because project d036a7b8844843af9f1103716b19f920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:03.001242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.001910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-ad3cad19-dac3-4079-a20a-97b0d4711024 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.003542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1178/2363] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:02 2022] DELETE /identity/v3/projects/d036a7b8844843af9f1103716b19f920 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:03.008715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.009231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.009231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.031545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:03.033562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:03.116155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1186/2364] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:03.127070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.127414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.127707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.148864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:03.150194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:03.159956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1179/2365] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:03.164615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.175048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d136d699-1fed-47d0-a8d3-82a9f70d7fed None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.180924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.181536 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.181813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.182081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.182733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:43:03.183039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.184761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.198035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1187/2366] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:43:03.202727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.213288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9853662-00d4-4965-906d-e13c8c36b085 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.218246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.218852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.219309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.219581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.220200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.222180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.253488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1180/2367] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:03.258207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.270112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56976a10-ca89-4d07-9319-0b9f01a0fda0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.315368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:03.334991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.335638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.335923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.336190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.336840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.338988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.362745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1188/2368] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:43:03.366959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.377934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba09d8-8087-44a0-8565-fc2d2d9ab14c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.383417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.384042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.384303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.384743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.385398 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.387105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.398380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1181/2369] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:03.402716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.413555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1cb0f-42c2-43ec-b17d-7bde823828d9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.419166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.419779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.420056 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.420353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.421018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.422635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a280f-553e-4d6b-aa21-f74897066044 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.434001 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1189/2370] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:03.438865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.448975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8421c7d-475c-49d6-9ec3-48a67e95db90 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.454067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.454750 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.455061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.455366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] PATH_INFO: `/v3/projects/ea6590ae0a5b4e8b8279597139c4e351/users/f7f5043fee384ddab379c5a56a83fa43/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.472019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea6590ae0a5b4e8b8279597139c4e351, user_id=f7f5043fee384ddab379c5a56a83fa43, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.475130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.490344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1182/2371] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:03 2022] PUT /identity/v3/projects/ea6590ae0a5b4e8b8279597139c4e351/users/f7f5043fee384ddab379c5a56a83fa43/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:03.501077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.501360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.501640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.530664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] MFA Rules not processed for user `f7f5043fee384ddab379c5a56a83fa43`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:03.531880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:03.600724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1190/2372] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:03.612755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.614523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:03.669197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:03.673535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba927fcb-4c6d-4abb-93ad-aca95ccbe5f3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.679414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.680576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.681051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.681474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.691942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.693427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03e80b-1a5a-4efd-a750-6f602d885ee6 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.702778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1183/2373] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:03.723359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:03.798739 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91af5747-c9aa-4ba3-b193-ce782a4f38e2 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:03.808031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:03.808913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.809309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.809581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] PATH_INFO: `/v3/users/f7f5043fee384ddab379c5a56a83fa43` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.811385 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f5043fee384ddab379c5a56a83fa43)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:03.814289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:03.871018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f5043fee384ddab379c5a56a83fa43'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.876653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:03.879357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f5043fee384ddab379c5a56a83fa43'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.880362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f5043fee384ddab379c5a56a83fa43'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.881770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f5043fee384ddab379c5a56a83fa43'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.892198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f5043fee384ddab379c5a56a83fa43'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:03.914464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1191/2374] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:03 2022] DELETE /identity/v3/users/f7f5043fee384ddab379c5a56a83fa43 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:03.919288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:03.919829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:03.919829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:03.940193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:03.941525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:04.008525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1184/2375] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:04.023694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.025178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:04.090026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cb1fcb-cd87-4bf5-93d4-492a885bc65c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.095915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.096906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.097247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.097590 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.113125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.114591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dde90a4-8cbb-4a49-9b87-2d983cde5ee4 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.125756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1192/2376] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:04.439329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.441185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:04.458225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d2e445-3ebb-42ac-b597-0524f1d1c21c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.465433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.466196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.466475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.466817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] PATH_INFO: `/v3/projects/ea6590ae0a5b4e8b8279597139c4e351` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.468241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea6590ae0a5b4e8b8279597139c4e351)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.470325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.523060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea6590ae0a5b4e8b8279597139c4e351', 'request_id': 'req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:04.529223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea6590ae0a5b4e8b8279597139c4e351', 'request_id': 'req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:04.538183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea6590ae0a5b4e8b8279597139c4e351', 'request_id': 'req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:04.553314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] The token cache is being invalidate because project ea6590ae0a5b4e8b8279597139c4e351 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:04.553742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:04.554723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b0a87ed7-b796-4627-ba0d-cba49edd0d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:04.556707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1185/2377] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:04 2022] DELETE /identity/v3/projects/ea6590ae0a5b4e8b8279597139c4e351 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:04.561707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.562105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.562465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.587397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:04.589186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:04.651498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1193/2378] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:04.662802 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.663054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.663386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.684660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:04.685908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:04.695438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1186/2379] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:04.699974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.713740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822e0759-bf43-4d12-a47f-c3cc6706f271 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.720855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.721675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.722042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.722437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.723283 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:43:04.723689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.726548 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.742124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1194/2380] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:43:04.748633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.762484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85caa782-8996-4cb7-b87c-10d2a9c31b76 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.769556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.770160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.770435 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.770684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.771331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.773238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.803252 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1187/2381] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:04 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:04.807725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.818308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bac1a-f04c-4011-9b08-78e002f2ab84 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.878330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.878928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.879238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.879513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.880123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.882046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.891138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:04.906446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1195/2382] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:04 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:43:04.912577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.927693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d830bf7-de74-46ff-b071-49651923a97f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.933282 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.934049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.934257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.934548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.935202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.937011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.947620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1188/2383] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:04.951973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.963494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4beb92-b1d9-4968-8dde-ed9db13fef91 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:04.969075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:04.969670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:04.969950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:04.970224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:04.970864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:04.972534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee967b-b078-43f2-a0f2-fb75d84f0828 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:04.984347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1196/2384] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:43:04.988499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:04.998704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb85475-66cf-46f8-80b0-b39af85865d5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:05.003684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:05.004329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:05.004608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:05.004881 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] PATH_INFO: `/v3/projects/cbbe8dca10314c80a4b1a3c7a623c45d/users/219418acdd1748b9a6292843e3dfaa0c/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:05.020203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbbe8dca10314c80a4b1a3c7a623c45d, user_id=219418acdd1748b9a6292843e3dfaa0c, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:05.023358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:05.040118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1189/2385] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:04 2022] PUT /identity/v3/projects/cbbe8dca10314c80a4b1a3c7a623c45d/users/219418acdd1748b9a6292843e3dfaa0c/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:05.050952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:05.051479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:05.051479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:05.080712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] MFA Rules not processed for user `219418acdd1748b9a6292843e3dfaa0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:05.081918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:05.160740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1197/2386] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:05.172236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:05.174399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:05.188877 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:05.235175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0854c1b-0a30-42ed-99b9-7ad48cf9150c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:05.239029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:05.239671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:05.239944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:05.240209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:05.248844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:05.250291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e9017-527d-4bc2-990c-83bf773833e9 service glance] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:05.259303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1190/2387] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:05 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:15.156765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:INSERT=2,keystone:SELECT=39 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:15.255359 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=21 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:32.579659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:32.705601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eba918-bbb9-4a08-8568-899223f15046 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:32.710828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:32.711646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:32.711928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:32.712202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] PATH_INFO: `/v3/users/219418acdd1748b9a6292843e3dfaa0c` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:32.713376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219418acdd1748b9a6292843e3dfaa0c)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:32.715379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:32.769606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219418acdd1748b9a6292843e3dfaa0c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:32.776875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219418acdd1748b9a6292843e3dfaa0c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:32.777594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219418acdd1748b9a6292843e3dfaa0c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:32.778223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219418acdd1748b9a6292843e3dfaa0c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:32.786111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219418acdd1748b9a6292843e3dfaa0c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:32.806905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1198/2388] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:32 2022] DELETE /identity/v3/users/219418acdd1748b9a6292843e3dfaa0c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:32.811536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:32.812124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:32.812124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:32.863516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:32.864809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:32.929605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1191/2389] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:32.939692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:32.941381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:32.991238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:33.056876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8186510-d571-4f8d-ae04-347dda4ec440 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.061170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.061825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.062139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.062625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.073549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.075804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc72332-1544-407e-96f9-e6e9e0d06ddd service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:33.087973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1199/2390] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:33.452550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:33.454505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:33.479054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22549ded-03b5-4c9a-84b6-57722f064acb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.493124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.494624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.495844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.496818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] PATH_INFO: `/v3/projects/cbbe8dca10314c80a4b1a3c7a623c45d` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.498949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbbe8dca10314c80a4b1a3c7a623c45d)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.502078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:33.568890 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbbe8dca10314c80a4b1a3c7a623c45d', 'request_id': 'req-4d20b4fd-02bd-47df-aed1-51cbd678f094'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:33.578612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbbe8dca10314c80a4b1a3c7a623c45d', 'request_id': 'req-4d20b4fd-02bd-47df-aed1-51cbd678f094'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:33.590251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbbe8dca10314c80a4b1a3c7a623c45d', 'request_id': 'req-4d20b4fd-02bd-47df-aed1-51cbd678f094'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:33.609017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] The token cache is being invalidate because project cbbe8dca10314c80a4b1a3c7a623c45d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:33.609450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:33.610309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:33.612237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1192/2391] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:33 2022] DELETE /identity/v3/projects/cbbe8dca10314c80a4b1a3c7a623c45d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:33.617037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d944-dd1d-4898-9361-52bc1159e051 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.617587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d944-dd1d-4898-9361-52bc1159e051 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.617694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d944-dd1d-4898-9361-52bc1159e051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.621238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:33.622950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:33.680088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cac6d944-dd1d-4898-9361-52bc1159e051 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:33.681418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cac6d944-dd1d-4898-9361-52bc1159e051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:33.707506 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:33.746215 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d20b4fd-02bd-47df-aed1-51cbd678f094 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.749680 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1200/2392] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:33.753245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.753910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.754185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.754453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.756323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:33.759350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.759706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.760045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.772400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.774245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:33.781406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:33.782760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:33.787892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1193/2393] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:43:33.791508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1201/2394] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:33.795797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:33.806753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07157b6-1543-4f82-ad1d-8ea660747dee service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.812266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.813645 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.814265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.814432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.815019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:43:33.815369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.817133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:33.831617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1194/2395] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:43:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:43:33.836753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:33.848763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74995df5-1a80-482b-9d1c-0f48327684ac None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.855176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.855841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.856111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.856380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.857015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.858955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:33.896664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1202/2396] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:33 2022] POST /identity/v3/projects => generated 359 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:43:33.898106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:33.910020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c249d571-a4bb-4fdc-86be-388604c7ce47 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:33.979545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:33.980173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:33.980721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:33.980850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:33.981443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:33.983564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.008503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1195/2397] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:33 2022] POST /identity/v3/users => generated 280 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:34.019998 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.030630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6776f4-adcb-4cc7-a935-82f49e580e20 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.035328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.035934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.036210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.036477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.037119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.038754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.049306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1203/2398] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:34.053496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.064704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082ad1dc-6f98-47c8-8877-093e520ec0f3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.070576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.071300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.071646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.071921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.072691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.074236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e6b6e2-a9d9-4ea6-b928-b3dead2b1664 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.086439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1196/2399] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:34.090919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.101888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04096748-16fe-4e6c-9f40-4f5c3809fab1 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.106460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.107160 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.107443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.107727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] PATH_INFO: `/v3/projects/c6aa9dd3b2d64c0d8c3e3d99a997c957/users/3c2278a522ed49aaaa3551492daf4737/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.122965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6aa9dd3b2d64c0d8c3e3d99a997c957, user_id=3c2278a522ed49aaaa3551492daf4737, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.125671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.140246 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1204/2400] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:34 2022] PUT /identity/v3/projects/c6aa9dd3b2d64c0d8c3e3d99a997c957/users/3c2278a522ed49aaaa3551492daf4737/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:34.214813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.215533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.215970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.264906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] MFA Rules not processed for user `3c2278a522ed49aaaa3551492daf4737`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:34.266179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:34.370360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1197/2401] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:34.409765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.412021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:34.434721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:34.480513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5504ba3-aecd-4822-9bc2-21bf52400c1e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.485552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.486213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.486485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.486752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.497011 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.498503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db6ae7-b9b5-423e-a275-1ee5c060aa9d service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.509813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1205/2402] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:34 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:34.525609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.553705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:34.595894 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114217f-236b-48d4-9e3c-2c843cb64c99 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.600955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.601638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.601932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.602196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] PATH_INFO: `/v3/users/3c2278a522ed49aaaa3551492daf4737` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.603372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2278a522ed49aaaa3551492daf4737)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.605319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.658461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2278a522ed49aaaa3551492daf4737'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:34.665776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2278a522ed49aaaa3551492daf4737'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:34.676491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2278a522ed49aaaa3551492daf4737'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:34.677247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2278a522ed49aaaa3551492daf4737'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:34.678021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2278a522ed49aaaa3551492daf4737'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:34.694922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1198/2403] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:34 2022] DELETE /identity/v3/users/3c2278a522ed49aaaa3551492daf4737 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:34.699621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.700177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.700177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.721738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:34.723376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:34.790479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1206/2404] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:34.806190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:34.808635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:34.884336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87768f94-ed53-4cb2-b7d5-d7b8358c5f2a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:34.888428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:34.889039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:34.889320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:34.889588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:34.901631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:34.903515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959e91-86e2-47c2-90a5-d9356bebce0b service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:34.915504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1199/2405] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:35.219250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.220764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:35.234714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7f989-4343-4432-9ed1-b03fa2fee46e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.240371 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.241120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.241458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.241797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] PATH_INFO: `/v3/projects/c6aa9dd3b2d64c0d8c3e3d99a997c957` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.243102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6aa9dd3b2d64c0d8c3e3d99a997c957)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.245182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.297082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6aa9dd3b2d64c0d8c3e3d99a997c957', 'request_id': 'req-134cef65-9be9-4439-a321-254a33c0a091'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:35.302738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6aa9dd3b2d64c0d8c3e3d99a997c957', 'request_id': 'req-134cef65-9be9-4439-a321-254a33c0a091'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:35.313926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6aa9dd3b2d64c0d8c3e3d99a997c957', 'request_id': 'req-134cef65-9be9-4439-a321-254a33c0a091'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:35.328167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] The token cache is being invalidate because project c6aa9dd3b2d64c0d8c3e3d99a997c957 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:35.328405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:35.329170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-134cef65-9be9-4439-a321-254a33c0a091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:35.330851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1207/2406] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:35 2022] DELETE /identity/v3/projects/c6aa9dd3b2d64c0d8c3e3d99a997c957 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:35.339940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.340329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.340841 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.365126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:35.366350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:35.425935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:35.455495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1200/2407] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:35.463664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.463951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.464222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.486188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:35.487469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:35.497786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1208/2408] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:35.502603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.513239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ca7c2a-d56f-49d9-8e16-9796bf503ff3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.518490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.519130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.519394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.519661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.520317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:43:35.520615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.522226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.535451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1201/2409] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:43:35.539965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.549517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e7164b-4400-494a-8328-b1ce66055ecd None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.554947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.555700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.556048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.556389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.557097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.559170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.603178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1209/2410] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:35.604543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.615768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f172659-22a9-462b-b0ba-3e12bd3b22ac admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.673886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.674508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.674779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.675045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.675714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.677710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.703145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1202/2411] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:35.704487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.716541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e16790-0da2-4473-bc07-88492d256823 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.723818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.724525 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.724873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.725211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.725943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.727862 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.743500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1210/2412] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:35.744801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.755433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd8f6b6-523a-42b8-b104-af7be81b2816 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.760513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.761119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.761397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.761667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.762317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.763985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36392970-ef46-4148-bedf-83ef0a141abb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.775621 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1203/2413] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:35.779740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:35.786838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:35.791913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454252ee-7fcd-4db4-8678-ee6ad281b22f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:35.798709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:35.799514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.799867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.800212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] PATH_INFO: `/v3/projects/6cfb3e6833c3428d97bc2b3fbbb9f67f/users/36e2345c777c48828d4f63805b05d71f/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.817662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfb3e6833c3428d97bc2b3fbbb9f67f, user_id=36e2345c777c48828d4f63805b05d71f, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:35.820603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:35.834855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1211/2414] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:35 2022] PUT /identity/v3/projects/6cfb3e6833c3428d97bc2b3fbbb9f67f/users/36e2345c777c48828d4f63805b05d71f/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:35.846536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:35.846821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:35.847118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:35.877307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] MFA Rules not processed for user `36e2345c777c48828d4f63805b05d71f`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:35.878550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:35.946085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1204/2415] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:35.950917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.023225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2965f949-3cd4-45ea-ad9e-340f4f53dc43 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.029446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.029840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.030112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.030390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.031030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.032994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.070750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1212/2416] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:35 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:36.073774 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.085030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ace8d-13ff-4361-8036-7b7743a6a13f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.134973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:36.162832 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.164141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.164141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.164141 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.164761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.166669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.194102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1205/2417] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:36 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:36.195432 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.209232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671382-55d7-4e6b-ae65-3641bcebf248 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.215701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.216374 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.216652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.216913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.217606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.219429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.232271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1213/2418] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:43:36.236684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.253622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668539a6-2022-4ce2-9c41-34c5d87dccf3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.259204 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.259886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.260164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.260438 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] PATH_INFO: `/v3/projects/ac3106f0cd4b4da59076ecda2526eb96/users/17f6e0924ef841cf85d26c03dfc4226f/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.277967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3106f0cd4b4da59076ecda2526eb96, user_id=17f6e0924ef841cf85d26c03dfc4226f, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.280884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.296459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1206/2419] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:36 2022] PUT /identity/v3/projects/ac3106f0cd4b4da59076ecda2526eb96/users/17f6e0924ef841cf85d26c03dfc4226f/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:36.300672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.313491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f696070-5844-458d-bcee-183a7938cb58 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.381969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.382818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.383334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.383660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.384522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.386158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a65bb-4ff0-4ede-93b4-23e4cc438127 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.397204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1214/2420] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:36.401477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.412653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10161e38-d288-44b3-be61-13fadc2e266a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.417502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.418419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.418703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.418977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] PATH_INFO: `/v3/projects/ac3106f0cd4b4da59076ecda2526eb96/users/17f6e0924ef841cf85d26c03dfc4226f/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.420927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3106f0cd4b4da59076ecda2526eb96, user_id=17f6e0924ef841cf85d26c03dfc4226f, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.423630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.441037 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1207/2421] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:36 2022] PUT /identity/v3/projects/ac3106f0cd4b4da59076ecda2526eb96/users/17f6e0924ef841cf85d26c03dfc4226f/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:36.452254 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1846a9-6b94-43e3-9c24-cb39adfb018c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.452563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1846a9-6b94-43e3-9c24-cb39adfb018c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.452843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1846a9-6b94-43e3-9c24-cb39adfb018c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.485204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2b1846a9-6b94-43e3-9c24-cb39adfb018c None None] MFA Rules not processed for user `17f6e0924ef841cf85d26c03dfc4226f`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:36.486436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2b1846a9-6b94-43e3-9c24-cb39adfb018c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:36.537713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:36.579735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1215/2422] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:36.592917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:36.595212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:36.662285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a657bdd-2b5a-4a8e-8edf-49cc478546f9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:36.666377 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:36.666998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:36.667380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:36.667643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:36.734595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:36.736111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:36.745253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1208/2423] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:37.461144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd5b8-a3f4-4640-95c5-bd614db4f027 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:37.461756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd5b8-a3f4-4640-95c5-bd614db4f027 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:37.461756 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bd5b8-a3f4-4640-95c5-bd614db4f027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:37.482880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0a6bd5b8-a3f4-4640-95c5-bd614db4f027 None None] MFA Rules not processed for user `fce6a7d7dbc94adeb0b03bae34e335a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:37.484185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0a6bd5b8-a3f4-4640-95c5-bd614db4f027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:37.545512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1216/2424] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:39.393280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:39.464731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:39.491102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d88e988-bb8e-4819-9d68-0076e3f1396f service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:39.497076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:39.497792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:39.498317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:39.498460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] PATH_INFO: `/v3/users/36e2345c777c48828d4f63805b05d71f` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:39.500009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e2345c777c48828d4f63805b05d71f)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:39.502358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:39.552673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e2345c777c48828d4f63805b05d71f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:39.559336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e2345c777c48828d4f63805b05d71f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:39.569256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e2345c777c48828d4f63805b05d71f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:39.569911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e2345c777c48828d4f63805b05d71f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:39.570644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e2345c777c48828d4f63805b05d71f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:39.591490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1209/2425] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:39 2022] DELETE /identity/v3/users/36e2345c777c48828d4f63805b05d71f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:39.597836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:39.598531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:39.598531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:39.618256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:39.619512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:39.693683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1217/2426] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:39.704471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:39.708214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:39.774994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517605b1-0b16-43a3-9a2d-04f6b377c9d7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:39.779412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:39.780071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:39.780346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:39.780613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:39.792197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:39.794155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:39.806703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1210/2427] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:40.108337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:40.110431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:40.136912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c4321f-9357-41c9-a8ef-2164e994ec1f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:40.143114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:40.143950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:40.144293 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:40.144624 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] PATH_INFO: `/v3/projects/6cfb3e6833c3428d97bc2b3fbbb9f67f` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:40.146123 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfb3e6833c3428d97bc2b3fbbb9f67f)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:40.148002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:40.216002 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfb3e6833c3428d97bc2b3fbbb9f67f', 'request_id': 'req-466a80f8-7d05-42ec-a13b-9cd29661e90b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.222906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfb3e6833c3428d97bc2b3fbbb9f67f', 'request_id': 'req-466a80f8-7d05-42ec-a13b-9cd29661e90b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.231100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfb3e6833c3428d97bc2b3fbbb9f67f', 'request_id': 'req-466a80f8-7d05-42ec-a13b-9cd29661e90b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.248637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] The token cache is being invalidate because project 6cfb3e6833c3428d97bc2b3fbbb9f67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:40.249006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.249663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.251630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1218/2428] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:40 2022] DELETE /identity/v3/projects/6cfb3e6833c3428d97bc2b3fbbb9f67f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:40.255402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:40.273554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5692c6-ce3f-46e8-9ba8-c46137c42e0d service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:40.353700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:40.354334 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:40.354612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:40.354878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] PATH_INFO: `/v3/users/17f6e0924ef841cf85d26c03dfc4226f` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:40.356050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f6e0924ef841cf85d26c03dfc4226f)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:40.357991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:40.370021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:40.424961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f6e0924ef841cf85d26c03dfc4226f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.434450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f6e0924ef841cf85d26c03dfc4226f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.444080 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f6e0924ef841cf85d26c03dfc4226f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.444837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f6e0924ef841cf85d26c03dfc4226f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.445769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-46978e67-a0b3-4387-b50f-5b67314c1fe1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f6e0924ef841cf85d26c03dfc4226f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:40.467959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1211/2429] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:43:40 2022] DELETE /identity/v3/users/17f6e0924ef841cf85d26c03dfc4226f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:40.896419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:40.899748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:40.910704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:40.918837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a80f8-7d05-42ec-a13b-9cd29661e90b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:40.996579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:40.997252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:40.997528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:40.997801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] PATH_INFO: `/v3/projects/ac3106f0cd4b4da59076ecda2526eb96` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:40.999014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3106f0cd4b4da59076ecda2526eb96)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.001050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.060296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3106f0cd4b4da59076ecda2526eb96', 'request_id': 'req-bd7b9803-7a53-4a0c-90e0-3522040e072c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:41.067977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3106f0cd4b4da59076ecda2526eb96', 'request_id': 'req-bd7b9803-7a53-4a0c-90e0-3522040e072c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:41.078510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3106f0cd4b4da59076ecda2526eb96', 'request_id': 'req-bd7b9803-7a53-4a0c-90e0-3522040e072c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:41.092945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] The token cache is being invalidate because project ac3106f0cd4b4da59076ecda2526eb96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:43:41.093267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:41.093957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bd7b9803-7a53-4a0c-90e0-3522040e072c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:43:41.095584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1219/2430] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:43:40 2022] DELETE /identity/v3/projects/ac3106f0cd4b4da59076ecda2526eb96 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:43:41.101576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.101869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.102135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.122863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:41.124195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:41.205196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1212/2431] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:41.216079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.216353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.216602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.253090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:41.254929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:41.275263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1220/2432] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:41.277927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.289268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d549a3c3-87a9-4f77-b35b-40d240d89fb5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.294574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.295257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.295537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.295942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.296620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:43:41.296925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.298616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.311983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1213/2433] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:43:41.317207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.329468 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8685d2-08f1-466b-a1c3-17efcf590542 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.336433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.337262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.337598 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.337970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.338837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.340792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.383380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1221/2434] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:43:41 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:43:41.384696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.395582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e40dcf-7bf0-4b9a-a1f3-59938094142a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.458389 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.459417 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.459745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.460085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.461002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.462951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.488280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1214/2435] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:43:41 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:43:41.492386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.505403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab63cd-48be-460b-aae0-703c4f809aa9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.511473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.512581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.512581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.512581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.513234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.514826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.525890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1222/2436] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:41.530023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.536689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:41.540461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32effc-ea1a-4ccf-8f30-8a512991751b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.546808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.547512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.548052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.548369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.549259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.551145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f75c2-abbd-4ebd-ba8f-d3c283aaa8e4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.562901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1215/2437] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:43:41.567478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.577987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eda0785-5a9f-49cf-b1a8-b3861f7dadc7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.584125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.584799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.585851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.586183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] PATH_INFO: `/v3/projects/a60666e80b524408a8f5662afe2f0868/users/5baa03ceeedb4e31bd78256486e55330/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.617290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a60666e80b524408a8f5662afe2f0868, user_id=5baa03ceeedb4e31bd78256486e55330, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.622237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.639457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1223/2438] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:43:41 2022] PUT /identity/v3/projects/a60666e80b524408a8f5662afe2f0868/users/5baa03ceeedb4e31bd78256486e55330/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:43:41.651362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bfb9a-ae9b-4b4e-95da-fe3af5d97287 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.651658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bfb9a-ae9b-4b4e-95da-fe3af5d97287 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.651932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144bfb9a-ae9b-4b4e-95da-fe3af5d97287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.685394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-144bfb9a-ae9b-4b4e-95da-fe3af5d97287 None None] MFA Rules not processed for user `5baa03ceeedb4e31bd78256486e55330`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:41.686755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-144bfb9a-ae9b-4b4e-95da-fe3af5d97287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:41.758139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1216/2439] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:41.769350 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:43:41.771150 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:41.812932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:41.888570 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b503f-abe3-400d-a14a-606e8622e3c5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:43:41.895359 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:43:41.896710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:41.898049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:41.898049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:41.912589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:43:41.914469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4f2b48-0986-4277-b5fe-6362a481ac9a service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:43:41.924484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1224/2440] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:43:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:43:43.319494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:43:43.320190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:43:43.320190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:43:43.339071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] MFA Rules not processed for user `5baa03ceeedb4e31bd78256486e55330`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:43:43.340311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:43:43.348042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1217/2441] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:43:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:43:51.920192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=21 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:43:53.334238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=38 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:01.341700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc63d0-3161-48ce-85fd-9cf60510ad8b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:01.342316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc63d0-3161-48ce-85fd-9cf60510ad8b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:01.342316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc63d0-3161-48ce-85fd-9cf60510ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:01.374984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c2cc63d0-3161-48ce-85fd-9cf60510ad8b None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:01.376295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c2cc63d0-3161-48ce-85fd-9cf60510ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:01.450062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1225/2442] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:44:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:01.506008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:01.507776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:01.587397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97de7a-5d4d-43bf-abb5-8c142b31b56b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:01.592742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:01.593376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:01.593653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:01.593923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:01.603712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:01.605354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:01.615505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1218/2443] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:02.222253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248a769-84a6-4aab-9395-f37672d8270c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:02.222629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248a769-84a6-4aab-9395-f37672d8270c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:02.222982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b248a769-84a6-4aab-9395-f37672d8270c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:02.245486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b248a769-84a6-4aab-9395-f37672d8270c None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:02.247217 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b248a769-84a6-4aab-9395-f37672d8270c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:02.255703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1226/2444] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:02.270208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:02.282120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989686b9-3cf0-42bf-8423-1d3238f318c0 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:02.287800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:02.288466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:02.288723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:02.289011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:02.298858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:02.300638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:02.310442 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1219/2445] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 13:44:02.570043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5829e-9c58-4995-bad8-032a849408b2 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:02.570668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5829e-9c58-4995-bad8-032a849408b2 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:02.570668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5829e-9c58-4995-bad8-032a849408b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:02.592134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2df5829e-9c58-4995-bad8-032a849408b2 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:02.593391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2df5829e-9c58-4995-bad8-032a849408b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:02.603387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1227/2446] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:44:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:44:02.620189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:02.631907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b729dfa0-513c-48f0-8ca8-38fbb54cbed6 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:02.637790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48965-0b12-4007-a383-562699ec485d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:02.638423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48965-0b12-4007-a383-562699ec485d service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:02.638700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48965-0b12-4007-a383-562699ec485d service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:02.638970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d48965-0b12-4007-a383-562699ec485d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:02.648690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d48965-0b12-4007-a383-562699ec485d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:02.650443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d48965-0b12-4007-a383-562699ec485d service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:02.660378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1220/2447] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:03.067838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:03.068414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:03.068898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:03.091707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:03.093031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:03.101259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1228/2448] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:44:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:03.114309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48965-0b12-4007-a383-562699ec485d service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:03.126499 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d48965-0b12-4007-a383-562699ec485d service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:03.132548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:03.133210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:03.133584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:03.133869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:03.144022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:03.145656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6dc7a9-ef61-4272-ab9f-875e83fc4b89 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:03.154936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1221/2449] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:05.477878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:05.545004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54aa7a1-73a2-490d-aa1c-6397300194f9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:05.549908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:05.550540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:05.550802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:05.551069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] PATH_INFO: `/v3/users/5baa03ceeedb4e31bd78256486e55330` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:05.552258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5baa03ceeedb4e31bd78256486e55330)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:05.554243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:05.612248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5baa03ceeedb4e31bd78256486e55330'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:05.621264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5baa03ceeedb4e31bd78256486e55330'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:05.622044 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5baa03ceeedb4e31bd78256486e55330'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:05.622773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5baa03ceeedb4e31bd78256486e55330'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:05.633759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5baa03ceeedb4e31bd78256486e55330'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:05.643554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:05.657992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1229/2450] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:44:05 2022] DELETE /identity/v3/users/5baa03ceeedb4e31bd78256486e55330 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:44:05.663986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:05.664803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:05.665296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:05.699542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:05.701239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:05.780862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1222/2451] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:44:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:05.791878 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:05.793352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:05.864243 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1408e9-eece-41c4-8d07-599d9e1443a0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:05.867913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:05.868531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:05.868811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:05.869090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:05.879547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:05.880774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9f828-f517-4299-b83b-ee1b8a19a168 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:05.891392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1230/2452] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:44:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:44:06.081346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:06.082915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:06.097017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a787a6-734e-4e41-a700-042c429cfbe5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:06.102083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:06.102801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:06.103090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:06.103379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] PATH_INFO: `/v3/projects/a60666e80b524408a8f5662afe2f0868` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:06.104868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=a60666e80b524408a8f5662afe2f0868)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:06.106775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:06.155069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:06.158038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a60666e80b524408a8f5662afe2f0868', 'request_id': 'req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:06.164761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a60666e80b524408a8f5662afe2f0868', 'request_id': 'req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:06.172541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a60666e80b524408a8f5662afe2f0868', 'request_id': 'req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:06.184140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] The token cache is being invalidate because project a60666e80b524408a8f5662afe2f0868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:44:06.184454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:06.185106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:44:06.186895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1223/2453] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:44:06 2022] DELETE /identity/v3/projects/a60666e80b524408a8f5662afe2f0868 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:44:09.879614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad7c3-de06-4465-b2c7-c7b61d1e87d9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:09.880264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad7c3-de06-4465-b2c7-c7b61d1e87d9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:09.880264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ad7c3-de06-4465-b2c7-c7b61d1e87d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:09.902611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4a8ad7c3-de06-4465-b2c7-c7b61d1e87d9 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:09.903929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4a8ad7c3-de06-4465-b2c7-c7b61d1e87d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:09.979219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1231/2454] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:44:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:09.993053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:09.994860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:10.081318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776bdb6c-b4af-4ac1-ac9c-7ac5fe7d95ce admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:10.089821 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:10.090593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:10.090875 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:10.091357 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:10.101770 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:10.103535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:10.113440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1224/2455] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:19.970324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=55,keystone:DELETE=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:20.109419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:28.351143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e6c3-05fb-46fc-acdd-5bdbada02cab None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:28.351660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e6c3-05fb-46fc-acdd-5bdbada02cab None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:28.351660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973e6c3-05fb-46fc-acdd-5bdbada02cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:28.373813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6973e6c3-05fb-46fc-acdd-5bdbada02cab None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:28.375032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6973e6c3-05fb-46fc-acdd-5bdbada02cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:28.384508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1232/2456] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:44:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:28.399947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:28.416286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8df56-21c7-45b1-9590-a78193d45eb2 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:28.423598 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:28.424207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:28.424508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:28.424785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:28.433866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:28.435492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:28.444053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1225/2457] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:38.368077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:38.440395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:48.199740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:48.200239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:48.200239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:48.220577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:44:48.221843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:44:48.229545 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1233/2458] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:44:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:44:48.242056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:44:48.253584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b6dd7-67f0-4688-8b61-879d8f138ce9 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:44:48.260024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:44:48.260665 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:44:48.260919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:44:48.261195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:44:48.270411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:44:48.272220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:44:48.282072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1226/2459] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:44:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:44:58.214527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:44:58.278251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:05.154823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:05.156604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:05.227127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19532387-ad7d-4503-998a-d00a9bd8f0b4 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:05.233994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:05.235477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:05.236141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:05.237136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:05.241040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:05.334404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:05.336237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:05.348149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1234/2460] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:09.325599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:09.328128 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:09.357050 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d3624-2737-48ca-a4f9-7b9fbca77aef service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:09.368032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:09.369329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:09.370056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:09.370763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] PATH_INFO: `/v3/users/651659e941044454985b2f866b791b07` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:09.401947 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=651659e941044454985b2f866b791b07)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:09.404078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:09.464745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '651659e941044454985b2f866b791b07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.474306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '651659e941044454985b2f866b791b07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.485162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '651659e941044454985b2f866b791b07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.485833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '651659e941044454985b2f866b791b07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.486685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-5b3f011e-775b-42d7-96da-a5d41d598dbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '651659e941044454985b2f866b791b07'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.508302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1227/2461] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:09 2022] DELETE /identity/v3/users/651659e941044454985b2f866b791b07 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:09.733046 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:09.746145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484339d-ad1d-4a09-aff4-5da27695b1e1 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:09.816026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:09.816735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:09.817050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:09.817446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] PATH_INFO: `/v3/projects/643240773a5840d3af2622469d696beb` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:09.830784 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=643240773a5840d3af2622469d696beb)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:09.833025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:09.884572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:09.891642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643240773a5840d3af2622469d696beb', 'request_id': 'req-4fc97be9-44e7-4afd-820e-879b356d71fc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.901426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643240773a5840d3af2622469d696beb', 'request_id': 'req-4fc97be9-44e7-4afd-820e-879b356d71fc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.912068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643240773a5840d3af2622469d696beb', 'request_id': 'req-4fc97be9-44e7-4afd-820e-879b356d71fc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.927918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] The token cache is being invalidate because project 643240773a5840d3af2622469d696beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:09.928363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.929212 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4fc97be9-44e7-4afd-820e-879b356d71fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:09.931032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1235/2462] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:09 2022] DELETE /identity/v3/projects/643240773a5840d3af2622469d696beb => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:09.936655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:09.936986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:09.937298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:09.959415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:09.960652 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:10.033372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1228/2463] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:45:10.042589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.042930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.043310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.065503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:10.066860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:10.077055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1236/2464] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:10.082008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.092613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1df9ac-de7e-4f4e-bd05-1d1e32398c9b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.098546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.099254 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.099530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.099799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.100482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:10.100785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.102459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.116362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1229/2465] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:45:10.121460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.131438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e3524-49db-4d6a-b383-898e88d1e956 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.136253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.136926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.137204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.137476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.138115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.140139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.182321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1237/2466] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:10 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:10.187301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.199035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e89a2-359c-476d-9f86-2ba3e7956481 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.256458 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:10.276023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.277524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.277777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.278052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.279346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.282205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.312836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1230/2467] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:10 2022] POST /identity/v3/users => generated 283 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:10.317149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.334387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15e8211-b9f5-446d-84bf-2445959aa8c5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.339542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.340170 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.340451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.340717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.341378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.343020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.355127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1238/2468] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:45:10.359842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.371895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b10121-c89b-479f-9774-3a77bc729d5c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.378097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.378738 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.379014 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.379749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.380428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.382103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff66129-29b5-4904-b65f-c5d5b833a566 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.397375 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1231/2469] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:10.402005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.413484 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71995237-befe-42b9-b0e0-dab3136e5628 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.419269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.419951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.420227 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.420494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] PATH_INFO: `/v3/projects/0f38cee660374729b5039d910379c5d6/users/2bdf80075aee483ebbdc0dfab07a3e39/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.439487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f38cee660374729b5039d910379c5d6, user_id=2bdf80075aee483ebbdc0dfab07a3e39, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.442959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.460517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1239/2470] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:10 2022] PUT /identity/v3/projects/0f38cee660374729b5039d910379c5d6/users/2bdf80075aee483ebbdc0dfab07a3e39/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:10.472390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.472694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.472969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.510104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] MFA Rules not processed for user `2bdf80075aee483ebbdc0dfab07a3e39`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:10.512403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:10.596342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1232/2471] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:10.608592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:10.610599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:10.705383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217a79d-78d0-470e-b17a-6918e131f411 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:10.711807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:10.712844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:10.713368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:10.713856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:10.730513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:10.733400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:10.752222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1240/2472] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:45:12.178764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:12.180425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:12.194512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71498c8-a408-46cf-9ee7-738ee8942beb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:12.199808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:12.200633 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:12.201122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:12.201493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:12.203833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:12.316294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:12.320489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:12.362388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1233/2473] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:13.535342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:13.544953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9381fa7-cb09-4322-a1ad-80b2165a9542 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:13.549596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:13.550218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:13.550500 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:13.550755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:13.552973 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:13.568516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:13.570015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c04dd-81ca-4b27-8ada-c088aed624c8 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:13.582815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1241/2474] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:19.655987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:19.684196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:19.726240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5254f28f-ff51-412d-a192-87036bb680e3 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:19.732819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:19.733920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:19.734304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:19.734667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] PATH_INFO: `/v3/users/2bdf80075aee483ebbdc0dfab07a3e39` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:19.737336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bdf80075aee483ebbdc0dfab07a3e39)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:19.741453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:19.795662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bdf80075aee483ebbdc0dfab07a3e39'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:19.803312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bdf80075aee483ebbdc0dfab07a3e39'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:19.811324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bdf80075aee483ebbdc0dfab07a3e39'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:19.812008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bdf80075aee483ebbdc0dfab07a3e39'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:19.812617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bdf80075aee483ebbdc0dfab07a3e39'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:19.835898 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1234/2475] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:19 2022] DELETE /identity/v3/users/2bdf80075aee483ebbdc0dfab07a3e39 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:19.841722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:19.842296 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:19.842406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:19.867494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:19.868760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:19.915297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:19.933375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1242/2476] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:19.943484 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:19.944998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:20.015277 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d0dc7-20ce-42af-98bc-5a706bf12f33 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.019234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.019850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.020143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.020501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.030687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.032172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb964b9f-b46f-4753-8c11-060849d1ea39 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.042065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1235/2477] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:20.326443 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.328154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:20.345568 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19aed0-95b1-407b-abac-20a6be45b24e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.351266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.351925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.352166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.352435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] PATH_INFO: `/v3/projects/0f38cee660374729b5039d910379c5d6` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.353595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f38cee660374729b5039d910379c5d6)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.355595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.418086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f38cee660374729b5039d910379c5d6', 'request_id': 'req-c41ccdc1-132a-4241-9652-f8912388df3e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:20.424474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f38cee660374729b5039d910379c5d6', 'request_id': 'req-c41ccdc1-132a-4241-9652-f8912388df3e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:20.436662 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f38cee660374729b5039d910379c5d6', 'request_id': 'req-c41ccdc1-132a-4241-9652-f8912388df3e'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:20.450884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] The token cache is being invalidate because project 0f38cee660374729b5039d910379c5d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:20.451065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:20.451789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c41ccdc1-132a-4241-9652-f8912388df3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:20.453380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1243/2478] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:20 2022] DELETE /identity/v3/projects/0f38cee660374729b5039d910379c5d6 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:20.458485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.458773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.459046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.480789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:20.481991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:20.524351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:20.546465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1236/2479] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:20.558417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.558769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.559203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.581433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:20.582654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:20.591798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1244/2480] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:20.596879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.611591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd45713-ff75-4c17-9aa2-6bba1ab1afcc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.616930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.617556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.617837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.618117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.618761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:20.619109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.620741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.634565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1237/2481] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:45:20.639194 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.649018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad9b0f-43ef-456d-a248-3abd9cc0a889 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.653999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.655047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.655619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.656076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.656983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.658915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.695015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1245/2482] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:20 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:20.699418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.714225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f21b57-6f74-4dff-ba9b-48cc459ad591 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.779786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.780414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.780666 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.780931 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.781570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.783582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.807312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1238/2483] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:20 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:20.812448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.824539 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210c7b56-ba2a-410b-94eb-6da5e5708c8e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.830971 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.831936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.832355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.832796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.833652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.835901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.847390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1246/2484] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:20.851647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.862545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e540e0-1d0b-4776-97c1-7661dcbbc1cf admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.868713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.869382 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.869748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.870024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.870673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.872323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec166813-fae8-4532-80f6-fb8c0f03acf8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.886108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1239/2485] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:20.890811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:20.903198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569bfd5-1279-471c-be6d-5462701b42a3 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:20.908589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:20.909669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.910075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.910340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] PATH_INFO: `/v3/projects/c624569a0dc744218ffa5d8c620fcf4b/users/1ccc3d2b354d467eb4c0074176b898f9/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.928435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c624569a0dc744218ffa5d8c620fcf4b, user_id=1ccc3d2b354d467eb4c0074176b898f9, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:20.932035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:20.949134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1247/2486] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:20 2022] PUT /identity/v3/projects/c624569a0dc744218ffa5d8c620fcf4b/users/1ccc3d2b354d467eb4c0074176b898f9/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:20.960682 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:20.960969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:20.961313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:20.990754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] MFA Rules not processed for user `1ccc3d2b354d467eb4c0074176b898f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:20.992262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:21.072217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1240/2487] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:45:21.077082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.142781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d892-0b10-4741-9d00-291b3d02fb77 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.148276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.148894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.149172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.149442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.150076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.152043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.169377 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:21.185139 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1248/2488] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:21 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:21.190202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.205434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7fc94b-dcb2-4c48-9523-16d9552642f6 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.253001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:21.282516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.283492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.284046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.284480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.285691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.289820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.319702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1241/2489] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:21 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:21.324928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.337381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077802f-916f-4b7a-bfce-ed2e3e335d40 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.343171 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.344153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.344429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.344699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.345349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.346989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.362124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1249/2490] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:21.366662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.380759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d39123-7210-4475-905d-e05160abf20b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.385699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.386390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.386677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.386954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] PATH_INFO: `/v3/projects/db3acf0d2f67459c806eb28553b911b1/users/a715587dc76349389631aaa7d4cdc71b/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.409299 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3acf0d2f67459c806eb28553b911b1, user_id=a715587dc76349389631aaa7d4cdc71b, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.412064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.434210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1242/2491] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:21 2022] PUT /identity/v3/projects/db3acf0d2f67459c806eb28553b911b1/users/a715587dc76349389631aaa7d4cdc71b/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:21.438585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.449470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04f697-53df-49e2-8107-9695f0b1de6d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.515673 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.515977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.516253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.516524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.517174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.518776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bd609-930d-4dcf-a48e-cdd6330280f6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.531055 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1250/2492] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:45:21.535459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.547190 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa4138-5dcb-4de2-9a24-8f9e393af4f3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.552105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.552759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.553033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.553365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] PATH_INFO: `/v3/projects/db3acf0d2f67459c806eb28553b911b1/users/a715587dc76349389631aaa7d4cdc71b/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.555324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3acf0d2f67459c806eb28553b911b1, user_id=a715587dc76349389631aaa7d4cdc71b, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.557978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.568993 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1243/2493] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:21 2022] PUT /identity/v3/projects/db3acf0d2f67459c806eb28553b911b1/users/a715587dc76349389631aaa7d4cdc71b/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:45:21.582040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.582340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.582584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.622520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] MFA Rules not processed for user `a715587dc76349389631aaa7d4cdc71b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:21.624632 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:21.726496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1251/2494] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:21 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:21.740255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:21.742676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:21.816563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71163cdc-2729-4d25-94d7-16dd400b1820 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:21.821146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:21.821813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:21.822195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:21.822528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:21.833678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:21.836434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4da953a-e9ad-42d3-93a0-fe44d26ebe07 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:21.849383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1244/2495] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:21 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:22.306321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:22.390050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232e0d4-1c14-461b-8a00-6eb9bcd3a327 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:22.395321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:22.396030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:22.396367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:22.396661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] PATH_INFO: `/v3/users/1ccc3d2b354d467eb4c0074176b898f9` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:22.397959 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ccc3d2b354d467eb4c0074176b898f9)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:22.399990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:22.405688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:22.464510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ccc3d2b354d467eb4c0074176b898f9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:22.473331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ccc3d2b354d467eb4c0074176b898f9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:22.474402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ccc3d2b354d467eb4c0074176b898f9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:22.475121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ccc3d2b354d467eb4c0074176b898f9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:22.484853 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ccc3d2b354d467eb4c0074176b898f9'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:22.515028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1252/2496] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:22 2022] DELETE /identity/v3/users/1ccc3d2b354d467eb4c0074176b898f9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:22.521858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:22.522180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:22.522597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:22.542303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:22.543836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:22.611537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1245/2497] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:22.621995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:22.623640 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:22.700984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db2211-ce35-4518-ab07-143553051f63 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:22.706387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:22.707109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:22.707387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:22.707681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:22.724768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:22.726434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:22.738063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1253/2498] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:23.077730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:23.078700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:23.094759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df9b1eb-c69e-460f-80ea-16adde73bb4e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:23.100003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:23.100836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:23.101179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:23.101520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] PATH_INFO: `/v3/projects/c624569a0dc744218ffa5d8c620fcf4b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:23.103211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c624569a0dc744218ffa5d8c620fcf4b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:23.106310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:23.156912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:23.177437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c624569a0dc744218ffa5d8c620fcf4b', 'request_id': 'req-4ee93f3b-14a9-49d2-a370-d17e2968986e'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.186082 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c624569a0dc744218ffa5d8c620fcf4b', 'request_id': 'req-4ee93f3b-14a9-49d2-a370-d17e2968986e'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.194451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c624569a0dc744218ffa5d8c620fcf4b', 'request_id': 'req-4ee93f3b-14a9-49d2-a370-d17e2968986e'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.210064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] The token cache is being invalidate because project c624569a0dc744218ffa5d8c620fcf4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:23.210319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.211363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.214739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1246/2499] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:23 2022] DELETE /identity/v3/projects/c624569a0dc744218ffa5d8c620fcf4b => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:23.224512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:23.234140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ffbf1-7aab-4336-ba5d-116aaf079ed9 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:23.326483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:23.327427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:23.327766 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:23.328050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] PATH_INFO: `/v3/users/a715587dc76349389631aaa7d4cdc71b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:23.329507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a715587dc76349389631aaa7d4cdc71b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:23.332535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:23.379319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:23.404325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a715587dc76349389631aaa7d4cdc71b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.412676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a715587dc76349389631aaa7d4cdc71b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.413559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a715587dc76349389631aaa7d4cdc71b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.414225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a715587dc76349389631aaa7d4cdc71b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.431395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b3e098b-c86d-42e4-9860-e925ad5644b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a715587dc76349389631aaa7d4cdc71b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.443058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1254/2500] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:23 2022] DELETE /identity/v3/users/a715587dc76349389631aaa7d4cdc71b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:23.761563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:23.762797 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:23.789451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee93f3b-14a9-49d2-a370-d17e2968986e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:23.891726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:23.892804 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:23.893273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:23.893751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] PATH_INFO: `/v3/projects/db3acf0d2f67459c806eb28553b911b1` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:23.895510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3acf0d2f67459c806eb28553b911b1)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:23.899504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:23.965531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3acf0d2f67459c806eb28553b911b1', 'request_id': 'req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.975450 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3acf0d2f67459c806eb28553b911b1', 'request_id': 'req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.985313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3acf0d2f67459c806eb28553b911b1', 'request_id': 'req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.997807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] The token cache is being invalidate because project db3acf0d2f67459c806eb28553b911b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:23.998111 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:23.998734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-cc3baea4-a44d-41eb-8cf2-7f19ed5b7cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:24.000279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1247/2501] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:23 2022] DELETE /identity/v3/projects/db3acf0d2f67459c806eb28553b911b1 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:24.005702 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.006015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.006285 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.028295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:24.029629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:24.115910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1255/2502] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:24.128175 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.128837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.128837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.151833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:24.153203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:24.162817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1248/2503] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:24.167578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.178511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b91b085-14f9-4d5e-b89e-820e3a896447 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.183883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.184518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.184829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.185102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.185783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:24.186144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.187894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.201429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1256/2504] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:45:24.207809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.219981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04a99a0-d05d-4dc4-bd1b-1401ea4c6deb None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.226136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.226855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.227269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.227712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.228643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.230671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.261433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1249/2505] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:24.266383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.278688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757197da-c525-417e-ab88-a3b0526686e4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.342806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.343563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.344089 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.344364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.345014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.346966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.370213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1257/2506] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:24.374728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.386152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb526861-18cd-43a5-b6d2-54a2ea6dd22e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.391427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.392062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.392342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.392613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.393278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.394897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.407454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1250/2507] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:24.411725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.422297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad419f39-7d73-436c-abea-10a780b8479b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.427593 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.428250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.428532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.428803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.429457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.431088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7079338c-b9b9-4bc6-89b0-e0210e24d2a3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.444679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1258/2508] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:24.450710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.462494 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6745ca-1152-4a85-8faa-83d3bde76fa9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.470946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.472042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.472548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.473044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] PATH_INFO: `/v3/projects/d3bc994355db4ccdb305499cf2a2b2be/users/023fb23f5ce64e0a9f33f31a618eb3b5/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.479679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:24.500186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3bc994355db4ccdb305499cf2a2b2be, user_id=023fb23f5ce64e0a9f33f31a618eb3b5, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.503553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.521225 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1251/2509] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:24 2022] PUT /identity/v3/projects/d3bc994355db4ccdb305499cf2a2b2be/users/023fb23f5ce64e0a9f33f31a618eb3b5/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:24.536161 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.536446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.536734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.563334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:24.575315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] MFA Rules not processed for user `023fb23f5ce64e0a9f33f31a618eb3b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:24.577972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:24.672322 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1259/2510] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:24.678243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.761826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4a669e-43ba-4fec-a568-4f164c5fcfa3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.769521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.770574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.771062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.771388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.772590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.776106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.819100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1252/2511] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/projects => generated 387 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:24.823565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.836487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d99309-f6b1-440f-bcc9-44fc28d3717e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.901666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.902325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.902630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.902941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.903628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.905796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.932697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1260/2512] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:24 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:24.934702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.946078 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7a028-75c4-409d-b0cd-7695cf8b7d53 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.951674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.952300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.952609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.952870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:24.953522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:24.955194 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:24.968162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1253/2513] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:24.974099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:24.988524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6956b-5857-41f9-8781-8bff5d380a82 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:24.996485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:24.997290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:24.997575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:24.997885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] PATH_INFO: `/v3/projects/39d3a61cb5524f89a5a532181a9c9193/users/85dd597f9e4a4f469e4ffc363f15965a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:25.018048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d3a61cb5524f89a5a532181a9c9193, user_id=85dd597f9e4a4f469e4ffc363f15965a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:25.022834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:25.043655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1261/2514] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:24 2022] PUT /identity/v3/projects/39d3a61cb5524f89a5a532181a9c9193/users/85dd597f9e4a4f469e4ffc363f15965a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:25.047701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:25.058272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2941018-39af-4378-a81c-1f6e72064241 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:25.139997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:25.141024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:25.141695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:25.142185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:25.143361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:25.145892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f21d2-c821-4601-b990-ae87039a19c5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:25.160561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1254/2515] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:25 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:25.165198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:25.176521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faf5fa9-e54b-494a-a37c-5c20146cd41c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:25.181692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:25.182367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:25.182647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:25.182913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] PATH_INFO: `/v3/projects/39d3a61cb5524f89a5a532181a9c9193/users/85dd597f9e4a4f469e4ffc363f15965a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:25.185127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=39d3a61cb5524f89a5a532181a9c9193, user_id=85dd597f9e4a4f469e4ffc363f15965a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:25.188235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:25.200264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1262/2516] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:25 2022] PUT /identity/v3/projects/39d3a61cb5524f89a5a532181a9c9193/users/85dd597f9e4a4f469e4ffc363f15965a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:25.212552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:25.212852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:25.213160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:25.255115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] MFA Rules not processed for user `85dd597f9e4a4f469e4ffc363f15965a`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:25.256733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:25.309232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:25.338392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1255/2517] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:25.350099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:25.352113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:25.418910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151565c2-59f7-49a3-a4e9-b03b33f272ce admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:25.423351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:25.424065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:25.424498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:25.424969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:25.433045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:25.437204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:25.439000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3d6f7-9162-4a36-9faa-e257b0e1b0ac service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:25.450481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1263/2518] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:26.179062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:26.262159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6bf5d-49a6-43d2-909b-49b0a851b301 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:26.267003 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:26.267776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:26.268057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:26.268330 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] PATH_INFO: `/v3/users/023fb23f5ce64e0a9f33f31a618eb3b5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:26.269522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=023fb23f5ce64e0a9f33f31a618eb3b5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:26.271582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:26.328983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '023fb23f5ce64e0a9f33f31a618eb3b5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:26.337163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '023fb23f5ce64e0a9f33f31a618eb3b5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:26.346529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '023fb23f5ce64e0a9f33f31a618eb3b5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:26.347318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '023fb23f5ce64e0a9f33f31a618eb3b5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:26.347936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '023fb23f5ce64e0a9f33f31a618eb3b5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:26.376658 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1256/2519] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:26 2022] DELETE /identity/v3/users/023fb23f5ce64e0a9f33f31a618eb3b5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:26.378518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:26.379024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:26.379024 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:26.399797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:26.401270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:26.489648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1264/2520] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:26.503974 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:26.506964 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:26.627184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0ffd5-dcc6-47c6-ab31-006a443b56fa admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:26.633622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:26.638168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:26.638441 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:26.638782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:26.649512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:26.651158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:26.662283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1257/2521] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:45:26.996553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:26.999251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:27.048460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da872-dc7c-4780-bee1-1a9b1b91721c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:27.060280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:27.060979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:27.067128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:27.067534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] PATH_INFO: `/v3/projects/d3bc994355db4ccdb305499cf2a2b2be` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:27.070237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3bc994355db4ccdb305499cf2a2b2be)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:27.075566 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:27.161449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3bc994355db4ccdb305499cf2a2b2be', 'request_id': 'req-4b760009-aa91-4ce2-94ab-be94d998d149'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.174513 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3bc994355db4ccdb305499cf2a2b2be', 'request_id': 'req-4b760009-aa91-4ce2-94ab-be94d998d149'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.191333 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3bc994355db4ccdb305499cf2a2b2be', 'request_id': 'req-4b760009-aa91-4ce2-94ab-be94d998d149'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.216903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] The token cache is being invalidate because project d3bc994355db4ccdb305499cf2a2b2be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:27.217286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.218079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.223392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1265/2522] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:26 2022] DELETE /identity/v3/projects/d3bc994355db4ccdb305499cf2a2b2be => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:27.225079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:27.231460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:27.241534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c089c-0211-4d8a-b0a5-e55b7f9ee00d service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:27.311516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:27.312166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:27.312447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:27.312721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] PATH_INFO: `/v3/users/85dd597f9e4a4f469e4ffc363f15965a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:27.313932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85dd597f9e4a4f469e4ffc363f15965a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:27.315970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:27.384100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85dd597f9e4a4f469e4ffc363f15965a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.394118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85dd597f9e4a4f469e4ffc363f15965a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.404563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85dd597f9e4a4f469e4ffc363f15965a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.405295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85dd597f9e4a4f469e4ffc363f15965a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.405929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8fd2783e-fd49-4a13-a563-9802a71a86c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85dd597f9e4a4f469e4ffc363f15965a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.431972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1258/2523] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:27 2022] DELETE /identity/v3/users/85dd597f9e4a4f469e4ffc363f15965a => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:27.780932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:27.783637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:27.801780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b760009-aa91-4ce2-94ab-be94d998d149 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:27.885558 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:27.886763 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:27.887042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:27.887719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] PATH_INFO: `/v3/projects/39d3a61cb5524f89a5a532181a9c9193` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:27.889149 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=39d3a61cb5524f89a5a532181a9c9193)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:27.891709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:27.926068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:27.973603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39d3a61cb5524f89a5a532181a9c9193', 'request_id': 'req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.981136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39d3a61cb5524f89a5a532181a9c9193', 'request_id': 'req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:27.992386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39d3a61cb5524f89a5a532181a9c9193', 'request_id': 'req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:28.006234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] The token cache is being invalidate because project 39d3a61cb5524f89a5a532181a9c9193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:28.007311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:28.008148 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9d34c99-2087-4b5d-89f1-45ccc17c5c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:28.009984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1266/2524] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:27 2022] DELETE /identity/v3/projects/39d3a61cb5524f89a5a532181a9c9193 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:28.018366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413ed3-2348-4483-a355-afb282484b11 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.018889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413ed3-2348-4483-a355-afb282484b11 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.019439 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57413ed3-2348-4483-a355-afb282484b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.047093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-57413ed3-2348-4483-a355-afb282484b11 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:28.048685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-57413ed3-2348-4483-a355-afb282484b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:28.126864 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1259/2525] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:28.140390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.141012 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.141503 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.171994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:28.173365 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:28.184392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1267/2526] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:28.188873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57413ed3-2348-4483-a355-afb282484b11 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.201130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57413ed3-2348-4483-a355-afb282484b11 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.206695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.207368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.207650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.207924 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.208597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:28.208963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.210731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.230790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1260/2527] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:45:28.231906 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.241497 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40596f9c-920d-4a5f-ab14-660ee506b198 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.246986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.247564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.247931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.248674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.248922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.250915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.285276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1268/2528] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:28 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:45:28.290468 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.301845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452f30c-b7a9-4af2-9c68-580891b6ec3a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.308246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:28.392759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.393502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.393787 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.394053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.394680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.397211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.446850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1261/2529] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:28 2022] POST /identity/v3/users => generated 287 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:28.451120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.477568 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca886f2-571f-424c-add7-e9bbda2c7200 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.483738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.484362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.484645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.484919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.485618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.487305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.507127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1269/2530] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:28.512350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.524934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f274e6c-3031-43ce-8afb-a8312c6b20ce admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.530515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.531729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.532279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.532758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.534108 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.537521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b769fb31-88f8-4439-a38a-5b759729ea4a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.554092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1262/2531] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:28.559688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.573426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcdaf00-e8d7-4a41-8dd5-e8a6d2cfdea2 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.579827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.580641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.580855 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.581165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] PATH_INFO: `/v3/projects/c5d72d32fa9e423c87fc597732205a7d/users/c03723dfafd84fe1afcbef41ec6ba481/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.598069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5d72d32fa9e423c87fc597732205a7d, user_id=c03723dfafd84fe1afcbef41ec6ba481, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.601043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.616718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1270/2532] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:28 2022] PUT /identity/v3/projects/c5d72d32fa9e423c87fc597732205a7d/users/c03723dfafd84fe1afcbef41ec6ba481/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:28.629452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.629772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.630004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.662435 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] MFA Rules not processed for user `c03723dfafd84fe1afcbef41ec6ba481`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:28.664028 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:28.763045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1263/2533] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:28.782247 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:28.784362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:28.866882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aeef05-4970-467e-be9c-29a78ceabcb4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:28.870929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:28.871613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:28.872210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:28.872210 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:28.885145 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:28.886882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a4dc4-6049-4d19-b553-58271b467e8f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:28.899851 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1271/2534] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:28.929365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:29.017309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44773f42-e2a0-4293-88a2-bce8bedb7a14 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:29.023351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:29.024065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.024332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.025773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] PATH_INFO: `/v3/users/c03723dfafd84fe1afcbef41ec6ba481` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:29.025773 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c03723dfafd84fe1afcbef41ec6ba481)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:29.027983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:29.059027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:29.116524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c03723dfafd84fe1afcbef41ec6ba481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.127868 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c03723dfafd84fe1afcbef41ec6ba481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.140379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c03723dfafd84fe1afcbef41ec6ba481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.141051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c03723dfafd84fe1afcbef41ec6ba481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.141715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c03723dfafd84fe1afcbef41ec6ba481'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.162223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1264/2535] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:28 2022] DELETE /identity/v3/users/c03723dfafd84fe1afcbef41ec6ba481 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:29.166955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.167284 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.167562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:29.201037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:29.208324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:29.265690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:29.286411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1272/2536] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:45:29.297758 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:29.299671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:29.369745 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c14edf-b27b-417c-a9b1-b85ec14e2bc8 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:29.373967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:29.374654 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.374932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.375263 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:29.386759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:29.388518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f079e0cd-6f7a-4e97-8434-1397705e908b service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:29.401319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1265/2537] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:29.727476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:29.729771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:29.746345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d7c72-0e95-4ecf-9181-e60573fffabc None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:29.754195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:29.755859 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.756504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.761788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] PATH_INFO: `/v3/projects/c5d72d32fa9e423c87fc597732205a7d` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:29.761788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5d72d32fa9e423c87fc597732205a7d)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:29.763498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:29.841498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5d72d32fa9e423c87fc597732205a7d', 'request_id': 'req-05a7e80e-1237-4921-8ac6-f06b78af19dc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.848164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5d72d32fa9e423c87fc597732205a7d', 'request_id': 'req-05a7e80e-1237-4921-8ac6-f06b78af19dc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.857899 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5d72d32fa9e423c87fc597732205a7d', 'request_id': 'req-05a7e80e-1237-4921-8ac6-f06b78af19dc'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.872469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] The token cache is being invalidate because project c5d72d32fa9e423c87fc597732205a7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:29.872469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.873440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-05a7e80e-1237-4921-8ac6-f06b78af19dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:29.875108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1273/2538] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:29 2022] DELETE /identity/v3/projects/c5d72d32fa9e423c87fc597732205a7d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:29.880281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.880566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.880816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:29.903392 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:29.904630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:29.970893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1266/2539] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:29.982370 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:29.982655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:29.982955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.005679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:30.006953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:30.016976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1274/2540] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:30.021699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.032855 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e09f1-7f4f-4caf-b6a0-c9fd3c10e5f9 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.039222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.040172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.040568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.040991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.041937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:30.042576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.045109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.059899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1267/2541] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:45:30.064724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.074541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f177d99-a357-4512-9023-bf52de8290a4 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.079619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.080230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.080509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.080776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.081411 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.083364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.116344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1275/2542] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:30 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:30.124294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.140353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629427b2-4ffa-4883-9511-b18c1bc2ac10 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.174248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:30.213976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.214603 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.214918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.215185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.215809 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.217749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.241788 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1268/2543] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:30 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:30.246065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.258368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c8664-ec08-4a82-aa79-ceb3bc60114a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.264132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.264857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.265248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.265528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.266253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.267993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.281509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1276/2544] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:30.285428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.295612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a089b-969d-4609-a53e-fd4e18e944d9 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.300739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.301369 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.301480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.301706 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.302360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.304044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71879483-5455-4d1e-8dab-ee9b8ced1444 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.314520 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1269/2545] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:45:30.319273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.329389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f75a51-02fb-4811-a427-3fe65b535043 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.334425 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.335127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.335395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.335644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] PATH_INFO: `/v3/projects/554bfc93b1a44d57a5e8e5868cd2eed6/users/14aab8af2e6b4201b0951199de8156e8/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.354521 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=554bfc93b1a44d57a5e8e5868cd2eed6, user_id=14aab8af2e6b4201b0951199de8156e8, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.357250 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.373829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1277/2546] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:30 2022] PUT /identity/v3/projects/554bfc93b1a44d57a5e8e5868cd2eed6/users/14aab8af2e6b4201b0951199de8156e8/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:30.379366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.379581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.379879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.407618 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:30.408856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:30.473008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1270/2547] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:30.482829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:30.484746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:30.547703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96453264-b904-4525-ad51-af41f2974392 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:30.552017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:30.552685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:30.552958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:30.553225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:30.564183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:30.565635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455d089b-9915-44b9-a49d-bc80a467d511 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:30.576478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1278/2548] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:33.038822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:33.040997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:33.059280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30deb5b-c0cc-4d64-afeb-f5f9db1f2b6e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:33.067920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:33.069069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:33.069683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:33.070046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:33.072358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:33.089273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:33.090678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:33.104607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1271/2549] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:45:36.508304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:36.509304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:36.509444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:36.559559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:36.572912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] MFA Rules not processed for user `14aab8af2e6b4201b0951199de8156e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:36.575052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:36.650188 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1279/2550] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:36.654985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:36.666214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac6a4-ee71-4d1c-ba0b-e09a0eafdb26 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:36.671179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:36.671778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:36.672056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:36.672319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:36.672949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:36.674860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:36.708736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1272/2551] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:36 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:45:36.713004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:36.726975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e4d52-a769-4b96-bc93-a999d8199238 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:36.837393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:36.838858 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:36.839355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:36.839755 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:36.842318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:36.845572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:36.874813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1280/2552] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:36 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:45:36.878350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:36.891794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e2c19-6889-4944-80ef-2d1712372bf4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:36.898344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:36.898720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:36.899244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:36.899691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:36.900562 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:36.902597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:36.917360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1273/2553] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:36.922189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:36.934430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0d47b-99e9-4fd8-8a9f-56390be9f6dc admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:36.939679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:36.940654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:36.941101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:36.941528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] PATH_INFO: `/v3/projects/c9163203450845d19ba4ccac40125db4/users/ce1aef1ef76f4d75a44eb7f3d74a70d8/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:36.959708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9163203450845d19ba4ccac40125db4, user_id=ce1aef1ef76f4d75a44eb7f3d74a70d8, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:36.962820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:36.986395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:36.987606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1281/2554] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:36 2022] PUT /identity/v3/projects/c9163203450845d19ba4ccac40125db4/users/ce1aef1ef76f4d75a44eb7f3d74a70d8/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:36.999242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a721a4-8a09-48d4-97a9-dc30c46752d4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:37.090372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:37.091363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:37.091490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:37.091938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:37.093530 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:37.095039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:37.096978 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:37.111275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1274/2555] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:36 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:37.113670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:37.125465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090fb67f-27a8-4eb6-befa-49b087860022 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:37.146429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:37.147552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:37.147923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:37.148294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] PATH_INFO: `/v3/projects/c9163203450845d19ba4ccac40125db4/users/ce1aef1ef76f4d75a44eb7f3d74a70d8/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:37.151588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9163203450845d19ba4ccac40125db4, user_id=ce1aef1ef76f4d75a44eb7f3d74a70d8, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:37.156915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:37.170426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1282/2556] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:37 2022] PUT /identity/v3/projects/c9163203450845d19ba4ccac40125db4/users/ce1aef1ef76f4d75a44eb7f3d74a70d8/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:37.816310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:37.820735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:37.935966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:37.940486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:38.031895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692108-b292-4e5f-8b73-324cb615cabc admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:38.047330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:38.070258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e1e45-b988-47a6-ac91-a0def5c4c57d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:38.075875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:38.076560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:38.076842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:38.077115 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] PATH_INFO: `/v3/users/fce6a7d7dbc94adeb0b03bae34e335a0` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:38.108256 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce6a7d7dbc94adeb0b03bae34e335a0)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:38.111382 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:38.118054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:38.119135 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:38.119734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:38.120046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:38.123869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:38.141753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:38.144212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:38.158332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1275/2557] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:38.185065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce6a7d7dbc94adeb0b03bae34e335a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.197022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce6a7d7dbc94adeb0b03bae34e335a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.198073 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce6a7d7dbc94adeb0b03bae34e335a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.198720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce6a7d7dbc94adeb0b03bae34e335a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.211809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-070f75a3-e538-4d53-9bf6-bde53a41d2c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce6a7d7dbc94adeb0b03bae34e335a0'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.237744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1283/2558] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:45:37 2022] DELETE /identity/v3/users/fce6a7d7dbc94adeb0b03bae34e335a0 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:38.543700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:38.560451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84af53-160e-42d3-9059-fe61afddf5ce service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:38.666785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:38.667604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:38.667893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:38.668170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] PATH_INFO: `/v3/projects/4293ff5f2cad4c399d03a89f4bc627f5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:38.693729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4293ff5f2cad4c399d03a89f4bc627f5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:38.696054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:38.765023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:38.843849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4293ff5f2cad4c399d03a89f4bc627f5', 'request_id': 'req-c85847d7-9995-4265-af3a-026ddd5ceccf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.860899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4293ff5f2cad4c399d03a89f4bc627f5', 'request_id': 'req-c85847d7-9995-4265-af3a-026ddd5ceccf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.877676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4293ff5f2cad4c399d03a89f4bc627f5', 'request_id': 'req-c85847d7-9995-4265-af3a-026ddd5ceccf'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.896585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] The token cache is being invalidate because project 4293ff5f2cad4c399d03a89f4bc627f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:45:38.896889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.897992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-c85847d7-9995-4265-af3a-026ddd5ceccf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:45:38.904160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1276/2559] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:45:38 2022] DELETE /identity/v3/projects/4293ff5f2cad4c399d03a89f4bc627f5 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:38.909836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:38.910213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:38.910560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:38.948316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:38.950898 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:39.111954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1284/2560] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:39.120374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c0776-8402-4556-b899-1992f4218afc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.120678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c0776-8402-4556-b899-1992f4218afc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.120957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c0776-8402-4556-b899-1992f4218afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.152638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-651c0776-8402-4556-b899-1992f4218afc None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:39.155372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-651c0776-8402-4556-b899-1992f4218afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:39.176319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1277/2561] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:39.181309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.192665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f62e79-dee9-4677-a015-b32035e7e0c7 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.198838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.199594 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.199953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.200298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.201318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:45:39.201751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.203591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.222405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1285/2562] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:45:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:45:39.223363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c0776-8402-4556-b899-1992f4218afc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.239256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c0776-8402-4556-b899-1992f4218afc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.252257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.253256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.253752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.254193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.255071 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.257430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.309187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1278/2563] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:39 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:45:39.313805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.326053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b8a674-c514-4fd0-9583-ea1d9ed42442 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.422058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:39.465488 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.466144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.466414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.466964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.467387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.470595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.507042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1286/2564] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:39 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:39.509360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.532038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a739506-2df2-4d0d-b1c6-b8aea8b9f122 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.545785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.546693 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.547309 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.548490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.549827 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.551799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.572690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1279/2565] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:39.577491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.596742 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f96269-5976-4d48-bb0e-c9ecb10ab69a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.605741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.606713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.607164 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.610162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.611543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.617660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd643c2-75e0-45f0-9c5b-1d42011e3f89 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.642528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1287/2566] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:39 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:39.644596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:39.671461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1839928-8d9b-4f10-a264-ac0bc874dbd7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:39.696268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:39.697176 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.697650 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.698105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] PATH_INFO: `/v3/projects/dff1f541b39e436e9f64df23295ce5e1/users/d98edbb359184b8f9d4c37533cf1c889/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.714606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dff1f541b39e436e9f64df23295ce5e1, user_id=d98edbb359184b8f9d4c37533cf1c889, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:39.717308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:39.747685 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1280/2567] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:39 2022] PUT /identity/v3/projects/dff1f541b39e436e9f64df23295ce5e1/users/d98edbb359184b8f9d4c37533cf1c889/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:45:39.769499 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96754370-074f-43f3-8baf-8cceca511756 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:39.769999 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96754370-074f-43f3-8baf-8cceca511756 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:39.770478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96754370-074f-43f3-8baf-8cceca511756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:39.819504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-96754370-074f-43f3-8baf-8cceca511756 None None] MFA Rules not processed for user `d98edbb359184b8f9d4c37533cf1c889`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:39.820992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-96754370-074f-43f3-8baf-8cceca511756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:39.932936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1288/2568] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:39.937571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.026094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d744c17-4062-4536-924b-e559016e2611 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.031543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.032268 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.032574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.032851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.033491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.035730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.091926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1281/2569] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:45:39 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:45:40.099343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96754370-074f-43f3-8baf-8cceca511756 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.116727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96754370-074f-43f3-8baf-8cceca511756 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.238077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.238077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.238077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.238077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.240226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.243642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.296704 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1289/2570] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:45:40 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:45:40.302247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.325167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbfff8b-615c-4a44-9d83-f6dfcd37a9fe admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.338418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.339226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.344448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.344863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.346587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.351394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.373137 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1282/2571] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:45:40.384029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.409458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee12835-949a-459b-8152-0140ca16280f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.422007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.422769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.423004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.423312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] PATH_INFO: `/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd/users/06f874495e8c46c78e5a36741e32bb70/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.475873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b0a382e6584ca79562dc8baec3fcdd, user_id=06f874495e8c46c78e5a36741e32bb70, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.479129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.514179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1290/2572] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:40 2022] PUT /identity/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd/users/06f874495e8c46c78e5a36741e32bb70/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:40.515872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.529926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013760d2-739e-493a-828d-1f6316534c7b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.579815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:40.681480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.682130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.682405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.682677 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.683429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.685125 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883171f-0469-40b0-a1a9-41ea24104ff0 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.697179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1283/2573] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:45:40.702486 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:40.720451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84fca82-de0f-45fd-ad71-7912003eb527 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:40.732917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:40.734459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.735047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.735600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] PATH_INFO: `/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd/users/06f874495e8c46c78e5a36741e32bb70/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.738299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9b0a382e6584ca79562dc8baec3fcdd, user_id=06f874495e8c46c78e5a36741e32bb70, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:40.741353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:40.763517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1291/2574] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:45:40 2022] PUT /identity/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd/users/06f874495e8c46c78e5a36741e32bb70/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:45:40.770640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:40.770994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:40.771476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:40.821925 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] MFA Rules not processed for user `06f874495e8c46c78e5a36741e32bb70`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:40.830697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:40.984043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1284/2575] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:45:41.003224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:41.008274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:41.031628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:41.157699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb7d533-456b-4e74-8ec0-b0b9dc5d3978 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:41.164900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:41.165867 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:41.166207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:41.166555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:41.185773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:41.188644 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b0af1-1bca-44ba-87e5-6a939c40be8a service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:41.212282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1292/2576] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:41.367989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:41.388916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3476f3-ba10-45c6-849f-dc67a86e00ab None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:41.397713 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:41.400531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:41.401639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:41.402963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:41.532535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:41.534089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:41.555186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1285/2577] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:43.564627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae2b-7e3c-4723-ab59-af171562da87 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:43.565239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae2b-7e3c-4723-ab59-af171562da87 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:43.565239 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094ae2b-7e3c-4723-ab59-af171562da87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:43.601795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3094ae2b-7e3c-4723-ab59-af171562da87 None None] MFA Rules not processed for user `ce1aef1ef76f4d75a44eb7f3d74a70d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:43.603128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3094ae2b-7e3c-4723-ab59-af171562da87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:43.719491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1293/2578] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:43.739253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:43.741717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:43.800122 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:43.817539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448258a6-b99a-4df6-aed6-9c4c38debcb8 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:43.821480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:43.822113 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:43.822490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:43.823687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:43.825107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:43.909035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:43.910587 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:43.922546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1286/2579] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:44.081355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224abb4-fa7c-4f2b-993e-b214478392b0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:44.081930 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224abb4-fa7c-4f2b-993e-b214478392b0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:44.082202 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d224abb4-fa7c-4f2b-993e-b214478392b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:44.140544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-d224abb4-fa7c-4f2b-993e-b214478392b0 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:44.141806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d224abb4-fa7c-4f2b-993e-b214478392b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:44.227301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1294/2580] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:45:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:44.256885 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:44.261915 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:44.384649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3bd7c-1877-4574-8153-4c024163e9b2 service glance] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:44.391298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:44.391935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:44.392951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:44.392951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:44.402604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:44.404320 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fae219-05ff-47be-89ad-7110dfe906de service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:44.415689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1287/2581] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:45:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:45:53.947985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:53.948463 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:53.948592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:53.968775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] MFA Rules not processed for user `14aab8af2e6b4201b0951199de8156e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:53.970554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:53.978107 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1295/2582] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:45:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:54.411995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=71 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:56.183946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:56.184557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:56.184557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:56.204622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:45:56.205880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:56.214393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1288/2583] 10.222.0.33 () {58 vars in 1009 bytes} [Mon Aug 8 13:45:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:45:56.231609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:56.238716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:45:56.246611 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6650f8-ae37-42d5-976e-47cfe22fdc2f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:56.255455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:56.256183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:56.256495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:56.256768 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:56.267599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:56.269490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:56.285783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1296/2584] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:45:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 08 13:45:57.187979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:57.190708 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:57.434635 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39fd92f-e289-43d5-beb2-ea5621fa86e6 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:57.456581 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:57.457272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:57.457664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:57.458247 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:57.462412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:57.477024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:57.479032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:57.491393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1289/2585] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:45:58.120302 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:45:58.134387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ee47ae-0a06-4a46-9aa2-e177a07ba2da service swift] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:45:58.146857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:45:58.148167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:45:58.148683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:45:58.149291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:45:58.152740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:45:58.313796 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:45:58.317786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:45:58.346700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1297/2586] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:45:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:04.235801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:04.238063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:04.261496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595b5faa-4a59-4b66-9c5f-c53aa8fa530b service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:04.268322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:04.270045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:04.270519 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:04.270860 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] PATH_INFO: `/v3/users/c29ce05ebc6e4672bb9557f751a48557` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:04.300959 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29ce05ebc6e4672bb9557f751a48557)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:04.303312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:04.365922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29ce05ebc6e4672bb9557f751a48557'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.374316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29ce05ebc6e4672bb9557f751a48557'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.383486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29ce05ebc6e4672bb9557f751a48557'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.384159 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29ce05ebc6e4672bb9557f751a48557'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.385465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b6508ea4-8858-4d3a-be60-dade405fe5fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29ce05ebc6e4672bb9557f751a48557'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.411485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1290/2587] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:04 2022] DELETE /identity/v3/users/c29ce05ebc6e4672bb9557f751a48557 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:04.625113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:04.636083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f9ca6-3214-417c-8822-7d91c3c7d978 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:04.708520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:04.709264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:04.709555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:04.709813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] PATH_INFO: `/v3/projects/0831512491044565b17fb6dd11a57b5f` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:04.722163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0831512491044565b17fb6dd11a57b5f)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:04.724151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:04.776103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0831512491044565b17fb6dd11a57b5f', 'request_id': 'req-351fafb3-cad0-4db7-9724-c3e06e80d6ef'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.781780 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0831512491044565b17fb6dd11a57b5f', 'request_id': 'req-351fafb3-cad0-4db7-9724-c3e06e80d6ef'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.789845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0831512491044565b17fb6dd11a57b5f', 'request_id': 'req-351fafb3-cad0-4db7-9724-c3e06e80d6ef'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.803864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] The token cache is being invalidate because project 0831512491044565b17fb6dd11a57b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:04.804223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.804932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-351fafb3-cad0-4db7-9724-c3e06e80d6ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:04.806483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1298/2588] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:04 2022] DELETE /identity/v3/projects/0831512491044565b17fb6dd11a57b5f => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:04.812607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:04.812889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:04.813164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:04.834201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:04.835504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:04.893134 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:04.907678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1291/2589] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:04.917070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:04.917496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:04.917774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:04.927939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:04.941805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:04.943048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:04.952387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1299/2590] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:04.957174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:04.967731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9916e2-2bf1-4beb-ae36-3f1eb8e7d925 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:04.973097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:04.973754 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:04.973994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:04.974290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:04.974951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:04.975294 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:04.976950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:04.991988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1292/2591] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:04.996597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.006912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be440e4-f173-4f6c-a514-d0e6604ef997 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.012090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.012712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.012987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.013255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.013918 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.015993 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.048641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1300/2592] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:04 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:05.053353 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.064343 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419977c3-1b0c-4ff5-a6e5-9c99d0c99ed4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.130231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.130869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.131188 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.131466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.132156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.134154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.159475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1293/2593] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:05 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:05.163949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.175376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584741-1dfb-4ad6-bbe3-b584150b9e6f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.180891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.181507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.181782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.182059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.182714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.184395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.196895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1301/2594] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:05.201463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.212515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc26524-7aa5-447d-a9c8-8834530593be admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.217614 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.218344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.218683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.219034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.220097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.221739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31209986-aa0c-490b-89d3-5680d76d7d53 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.233509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1294/2595] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:05.238271 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.250069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0e879-f791-4b81-b907-10971cab3f67 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.255304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.255997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.256261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.256540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] PATH_INFO: `/v3/projects/917734263ab541259e07184baad474a9/users/bd75e4e3f9a04f339418415cdc4760c9/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.274441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=917734263ab541259e07184baad474a9, user_id=bd75e4e3f9a04f339418415cdc4760c9, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.277157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.299101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1302/2596] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:05 2022] PUT /identity/v3/projects/917734263ab541259e07184baad474a9/users/bd75e4e3f9a04f339418415cdc4760c9/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:05.310878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.310878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.310878 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.347428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] MFA Rules not processed for user `bd75e4e3f9a04f339418415cdc4760c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:05.348777 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:05.437913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1295/2597] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:05.439035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.534381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fe075-0ae9-4557-bb40-e414ddf03d6f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.540440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.541147 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.541575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.541921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.542659 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.544740 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.576708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1303/2598] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:05 2022] POST /identity/v3/projects => generated 369 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:05.581286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.604009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ce65eb-f8de-4d68-a62a-4b1c319e0669 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.666922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:05.681917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.682757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.683167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.683561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.684486 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.689946 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.722973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1296/2599] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:05 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:05.727623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.739788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c407ec-51fc-4350-9a20-fa0716dd4c88 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.746129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.750689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.751265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.751786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.752471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.754133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.766438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1304/2600] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:05.771661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.789583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0078e-7d2c-4359-bda8-35306feaab79 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.797555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.798690 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.799217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.799721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] PATH_INFO: `/v3/projects/3db185a75c6f4a0aae28196266a922c1/users/4a148ac22f3d493b86480ca80321ef71/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.818127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db185a75c6f4a0aae28196266a922c1, user_id=4a148ac22f3d493b86480ca80321ef71, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.821279 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.836987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1297/2601] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:05 2022] PUT /identity/v3/projects/3db185a75c6f4a0aae28196266a922c1/users/4a148ac22f3d493b86480ca80321ef71/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:05.842305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.858989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7414992-636a-485e-a69a-f3b1ff466cc7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.922270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.922896 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.923223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.923502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.924162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.925773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f76c99-9553-4a0f-b2b7-4b01180c8956 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.936394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1305/2602] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:05 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:05.941083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:05.957529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295d590-ad16-410c-b748-be192771bbf4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:05.964052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:05.964751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:05.965077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:05.965541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] PATH_INFO: `/v3/projects/3db185a75c6f4a0aae28196266a922c1/users/4a148ac22f3d493b86480ca80321ef71/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:05.968564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=3db185a75c6f4a0aae28196266a922c1, user_id=4a148ac22f3d493b86480ca80321ef71, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:05.971763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:05.985870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1298/2603] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:05 2022] PUT /identity/v3/projects/3db185a75c6f4a0aae28196266a922c1/users/4a148ac22f3d493b86480ca80321ef71/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:06.002041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:06.002491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:06.002892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:06.025599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:06.035735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] MFA Rules not processed for user `4a148ac22f3d493b86480ca80321ef71`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:06.037341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:06.115323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1306/2604] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:06 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:06.127555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:06.129455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:06.199966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5ae2b4-5ad0-4166-8771-a83c0a26fdfc admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:06.203619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:06.204236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:06.204510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:06.204783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:06.216090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:06.217558 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:06.229020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1299/2605] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:06.279749 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:06.281441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:06.305925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8f0d93-09fb-4919-b046-84cd6deba41b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:06.317431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:06.318075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:06.318355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:06.318626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:06.393264 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:06.394782 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ebcd9-8500-41ad-bd6e-aba8995e06d4 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:06.405134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1307/2606] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:09.916238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:09.986386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de63df-eb80-4d44-b44f-df12fd7b27ca service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:09.992542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:09.993210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:09.993490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:09.993753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] PATH_INFO: `/v3/users/bd75e4e3f9a04f339418415cdc4760c9` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:09.995103 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd75e4e3f9a04f339418415cdc4760c9)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:09.997069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:10.059767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd75e4e3f9a04f339418415cdc4760c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.061980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:10.069422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd75e4e3f9a04f339418415cdc4760c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.079505 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd75e4e3f9a04f339418415cdc4760c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.080132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd75e4e3f9a04f339418415cdc4760c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.080769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd75e4e3f9a04f339418415cdc4760c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.100586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1300/2607] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:09 2022] DELETE /identity/v3/users/bd75e4e3f9a04f339418415cdc4760c9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:10.107821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:10.108389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:10.108552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:10.130638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:10.132069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:10.197280 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1308/2608] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:10.207891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:10.209466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:10.288662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85426a6a-b1cd-49fb-92f5-6151b036700c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:10.293829 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:10.294524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:10.294911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:10.295323 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:10.307002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:10.308565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:10.319695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1301/2609] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:46:10.672776 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:10.674651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:10.693218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61794d25-8783-41a0-ad13-0ff18aa2f333 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:10.702157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:10.703332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:10.703769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:10.704134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] PATH_INFO: `/v3/projects/917734263ab541259e07184baad474a9` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:10.706183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=917734263ab541259e07184baad474a9)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:10.708709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:10.748292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:10.765844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '917734263ab541259e07184baad474a9', 'request_id': 'req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.772444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '917734263ab541259e07184baad474a9', 'request_id': 'req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.783716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '917734263ab541259e07184baad474a9', 'request_id': 'req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.797495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] The token cache is being invalidate because project 917734263ab541259e07184baad474a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:10.797995 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.802805 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:10.804687 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1309/2610] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:10 2022] DELETE /identity/v3/projects/917734263ab541259e07184baad474a9 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:10.812495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:10.824222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d93bc-2ede-4d88-9795-2b58d629e65a service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:10.901602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:10.902739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:10.903262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:10.903793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] PATH_INFO: `/v3/users/4a148ac22f3d493b86480ca80321ef71` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:10.905934 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a148ac22f3d493b86480ca80321ef71)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:10.909716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:10.992061 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a148ac22f3d493b86480ca80321ef71'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.001205 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a148ac22f3d493b86480ca80321ef71'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.010869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a148ac22f3d493b86480ca80321ef71'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.011644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a148ac22f3d493b86480ca80321ef71'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.012269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-f7a7a4bb-7359-41eb-a4e2-ffe33f7461cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a148ac22f3d493b86480ca80321ef71'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.020609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:11.037451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1302/2611] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:10 2022] DELETE /identity/v3/users/4a148ac22f3d493b86480ca80321ef71 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:11.384555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:11.386082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:11.401057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295e7d7f-b0ac-4e36-9e7a-1704a58c0cfa admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:11.464020 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:11.464964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.465319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.465596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] PATH_INFO: `/v3/projects/3db185a75c6f4a0aae28196266a922c1` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.467608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3db185a75c6f4a0aae28196266a922c1)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:11.470442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:11.538928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3db185a75c6f4a0aae28196266a922c1', 'request_id': 'req-39d679e7-8e5d-4174-b547-8b26c1f1add5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.545085 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3db185a75c6f4a0aae28196266a922c1', 'request_id': 'req-39d679e7-8e5d-4174-b547-8b26c1f1add5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.554248 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3db185a75c6f4a0aae28196266a922c1', 'request_id': 'req-39d679e7-8e5d-4174-b547-8b26c1f1add5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.568495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] The token cache is being invalidate because project 3db185a75c6f4a0aae28196266a922c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:11.568870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.569778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-39d679e7-8e5d-4174-b547-8b26c1f1add5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:11.571504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1310/2612] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:11 2022] DELETE /identity/v3/projects/3db185a75c6f4a0aae28196266a922c1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:11.576989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.577462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.577867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.604503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:11.605751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:11.679543 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1303/2613] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:11.687511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.687792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.688088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.709048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:11.710410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:11.719426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1311/2614] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:11.727327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:11.737631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dbe93-177b-4c2b-bbae-4d1009c27e76 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:11.742596 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:11.743259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.743539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.743810 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.744605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:11.745009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:11.746649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:11.759656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1304/2615] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:11.764345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:11.776190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c281cc3a-7a22-4461-a38f-11e90b2a0a74 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:11.782565 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:11.783230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.783506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.783795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.784429 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:11.786348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:11.827974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1312/2616] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:11 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:11.831655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:11.848971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb9f0d-414c-4a1e-87e3-7c1d3a63af9e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:11.935396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:11.936010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.936284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.936557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.937197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:11.939149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:11.962400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1305/2617] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:11 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:11.967298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:11.980184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d4871-319e-46dc-b29d-aa7270ac17b0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:11.988712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:11.989592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:11.989947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:11.990251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:11.991192 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:11.993104 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.004682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1313/2618] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:12.010286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.020937 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb1f8e-b9a6-48a8-aca5-004222e65756 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.026088 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.026697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.027092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.027274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.027927 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.029536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3dad06-a3c2-4e90-b248-4dac93350d4d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.040872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1306/2619] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:12.045653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.056716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61662f95-0869-4dd7-8004-28c521ef3584 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.061975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.062642 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.062921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.063286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] PATH_INFO: `/v3/projects/7a070e86d7574daf9aa36c5d9d434993/users/16d690db050b4f109f6b02a2cf17ff0d/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.066717 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:12.080743 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a070e86d7574daf9aa36c5d9d434993, user_id=16d690db050b4f109f6b02a2cf17ff0d, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.083677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.100281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1314/2620] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:12 2022] PUT /identity/v3/projects/7a070e86d7574daf9aa36c5d9d434993/users/16d690db050b4f109f6b02a2cf17ff0d/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:12.111704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.112043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.112349 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.143032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] MFA Rules not processed for user `16d690db050b4f109f6b02a2cf17ff0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:12.144301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:12.193534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:12.217242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1307/2621] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:12.222860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.290824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc892f-2f0c-4a68-bd14-1967a11bda01 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.296547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.297211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.297533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.297748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.298391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.300368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.330128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1315/2622] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:12.334921 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.349015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41134e1-fc14-4925-b059-d462aab2e516 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.410264 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.410884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.411200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.411476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.412107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.414089 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.439123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1308/2623] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/users => generated 294 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:12.443936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.458951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521636c-6638-4f42-bbc4-fbca4d115b5b admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.464849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.465467 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.465741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.466008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.466661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.468563 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.480895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1316/2624] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:12.485171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.497228 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1207c57-2c54-4e02-8801-ca1c4a0ca11d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.502752 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.503407 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.503684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.503956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.504629 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.506461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e158ee2-e480-4871-b0e7-b713fbc4d29c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.517825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1309/2625] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:12.523048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.536027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88bea82-142c-4422-a460-70bd6693f216 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.541981 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.542646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.542926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.543237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] PATH_INFO: `/v3/projects/50490cef51da4d16a9a1a23cad1d1434/users/e3e8436e2978410cbfc7aaf3567984b7/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.559826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50490cef51da4d16a9a1a23cad1d1434, user_id=e3e8436e2978410cbfc7aaf3567984b7, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.562478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.577936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1317/2626] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:12 2022] PUT /identity/v3/projects/50490cef51da4d16a9a1a23cad1d1434/users/e3e8436e2978410cbfc7aaf3567984b7/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:12.590508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.590734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.590950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.625411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] MFA Rules not processed for user `e3e8436e2978410cbfc7aaf3567984b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:12.626644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:12.698312 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1310/2627] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:12.703111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.773549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b3db7-dcc2-452f-be4a-a9b19723ce02 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.782441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.783117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.783476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.783752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.784393 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.786402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.820802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1318/2628] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:12.822232 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.834060 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd7fc0c-d655-4588-ae7f-a3cd986efc57 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.899001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:12.911356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.911983 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.912266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.912541 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.913181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.915361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.942311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1311/2629] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:12 2022] POST /identity/v3/users => generated 300 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:12.943838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.954868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d7a1a-7a87-4221-8f5e-82fb094da9f6 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.959577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:12.960190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:12.960498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:12.960779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:12.961445 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:12.963057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:12.975326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1319/2630] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:12.981193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:12.992401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4945ad8-6886-4306-b4d8-cb047afeae36 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:12.999768 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:13.000527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:13.000811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:13.001153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] PATH_INFO: `/v3/projects/5609eb7cbba745a886d59a10ebf92dac/users/52e0587e31d545c5a3ad9ac0a55a8173/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:13.018801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5609eb7cbba745a886d59a10ebf92dac, user_id=52e0587e31d545c5a3ad9ac0a55a8173, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:13.021508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:13.036074 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1312/2631] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:12 2022] PUT /identity/v3/projects/5609eb7cbba745a886d59a10ebf92dac/users/52e0587e31d545c5a3ad9ac0a55a8173/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:13.040235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:13.050802 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af34ca-f6d8-4b8b-a1f7-235825d6301e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:13.055681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:13.110320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:13.111312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:13.111665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:13.112000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:13.113059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:13.115676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f1ff-a5bf-4f9e-ab15-a19f87bff56f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:13.126261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1320/2632] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:13 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:13.130683 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:13.146083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8bbfea-f011-4573-b6e0-01e318ca23d4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:13.154767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:13.155786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:13.156221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:13.156565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] PATH_INFO: `/v3/projects/5609eb7cbba745a886d59a10ebf92dac/users/52e0587e31d545c5a3ad9ac0a55a8173/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:13.160022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5609eb7cbba745a886d59a10ebf92dac, user_id=52e0587e31d545c5a3ad9ac0a55a8173, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:13.164413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:13.178741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1313/2633] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:13 2022] PUT /identity/v3/projects/5609eb7cbba745a886d59a10ebf92dac/users/52e0587e31d545c5a3ad9ac0a55a8173/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:13.189077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:13.189321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:13.189649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:13.219492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] MFA Rules not processed for user `52e0587e31d545c5a3ad9ac0a55a8173`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:13.221111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:13.294240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1321/2634] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:13 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:13.305886 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:13.307329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:13.371913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d57ef5-e834-424e-92d8-acf46931d048 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:13.376143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:13.376772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:13.377046 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:13.377316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:13.387895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:13.389365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:13.400600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1314/2635] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:13 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:13.976535 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:13.978141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:13.991908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64fb36-3d7f-4502-aff8-414c32e7ebb4 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:14.004039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:14.005008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:14.005325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:14.005600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:14.078707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:14.080096 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:14.091817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1322/2636] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:13 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:19.356729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:19.366858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f54c16c-38f7-4cb5-b3d4-deee618e8efe service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:19.371163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:19.371775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:19.372055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:19.372324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:19.440643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:19.442106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26c185-736b-49eb-be9d-266b6a76cea6 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:19.452428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1315/2637] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:21.716351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:21.891893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=100 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:21.937196 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bc6e-1648-40f8-986b-ef8440b74c13 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:21.942903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:21.943734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:21.944068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:21.944509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] PATH_INFO: `/v3/users/16d690db050b4f109f6b02a2cf17ff0d` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:21.946184 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d690db050b4f109f6b02a2cf17ff0d)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:21.948821 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:22.005029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d690db050b4f109f6b02a2cf17ff0d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.012222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d690db050b4f109f6b02a2cf17ff0d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.012891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d690db050b4f109f6b02a2cf17ff0d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.013491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d690db050b4f109f6b02a2cf17ff0d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.021358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d690db050b4f109f6b02a2cf17ff0d'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.039345 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1323/2638] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:21 2022] DELETE /identity/v3/users/16d690db050b4f109f6b02a2cf17ff0d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:22.044593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:22.045160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:22.045659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:22.075987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:22.077227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:22.130717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:22.150513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1316/2639] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:22.161589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:22.164125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:22.234835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a8c6f3-b925-4373-9953-e1f1738453e8 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:22.238887 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:22.239547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:22.239826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:22.240097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:22.250174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:22.251665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:22.262701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1324/2640] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:22.558566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:22.560321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:22.577367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b3ef58-b0fa-4172-ae07-b974d4c9487e None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:22.584197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:22.584836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:22.585112 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:22.585513 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] PATH_INFO: `/v3/projects/7a070e86d7574daf9aa36c5d9d434993` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:22.586919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a070e86d7574daf9aa36c5d9d434993)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:22.589466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:22.639293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a070e86d7574daf9aa36c5d9d434993', 'request_id': 'req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.645324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a070e86d7574daf9aa36c5d9d434993', 'request_id': 'req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.654055 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a070e86d7574daf9aa36c5d9d434993', 'request_id': 'req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.665663 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] The token cache is being invalidate because project 7a070e86d7574daf9aa36c5d9d434993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:22.665951 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.666576 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.668166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1317/2641] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:22 2022] DELETE /identity/v3/projects/7a070e86d7574daf9aa36c5d9d434993 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:22.672461 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:22.682218 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3755126-ffea-49c3-a269-4921e29fb668 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:22.730154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:22.744433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:22.745356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:22.745746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:22.746136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] PATH_INFO: `/v3/users/e3e8436e2978410cbfc7aaf3567984b7` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:22.747915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3e8436e2978410cbfc7aaf3567984b7)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:22.750892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:22.801940 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3e8436e2978410cbfc7aaf3567984b7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.808863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3e8436e2978410cbfc7aaf3567984b7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.809526 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3e8436e2978410cbfc7aaf3567984b7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.810126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3e8436e2978410cbfc7aaf3567984b7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.818562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3e8436e2978410cbfc7aaf3567984b7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:22.839779 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1325/2642] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:22 2022] DELETE /identity/v3/users/e3e8436e2978410cbfc7aaf3567984b7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:23.135313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:23.137012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:23.150900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c0bdb-56d0-4ef3-ba4b-2e8271c7bf83 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:23.209547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:23.210105 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:23.210379 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:23.210649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] PATH_INFO: `/v3/projects/50490cef51da4d16a9a1a23cad1d1434` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:23.211879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] RBAC: Authorizing `identity:delete_project(project_id=50490cef51da4d16a9a1a23cad1d1434)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:23.213760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:23.264127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50490cef51da4d16a9a1a23cad1d1434', 'request_id': 'req-dac666f3-36d4-4e65-a985-b93eda8d5ffd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.270496 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50490cef51da4d16a9a1a23cad1d1434', 'request_id': 'req-dac666f3-36d4-4e65-a985-b93eda8d5ffd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.280139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50490cef51da4d16a9a1a23cad1d1434', 'request_id': 'req-dac666f3-36d4-4e65-a985-b93eda8d5ffd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.294437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:23.301620 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] The token cache is being invalidate because project 50490cef51da4d16a9a1a23cad1d1434 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:23.302143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.303373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.306386 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1318/2643] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:23 2022] DELETE /identity/v3/projects/50490cef51da4d16a9a1a23cad1d1434 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:23.312372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:23.314274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:23.330051 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dd7e0-c677-4a6e-8e47-98eccfe541ef admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:23.398532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:23.399230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:23.399510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:23.399777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] PATH_INFO: `/v3/users/52e0587e31d545c5a3ad9ac0a55a8173` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:23.401559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52e0587e31d545c5a3ad9ac0a55a8173)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:23.404225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:23.463504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52e0587e31d545c5a3ad9ac0a55a8173'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.471891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52e0587e31d545c5a3ad9ac0a55a8173'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.472616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52e0587e31d545c5a3ad9ac0a55a8173'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.473292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52e0587e31d545c5a3ad9ac0a55a8173'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.483142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-339a201d-1aef-4a3b-9b71-ba45dd84f72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52e0587e31d545c5a3ad9ac0a55a8173'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.491389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:23.506216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1326/2644] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:23 2022] DELETE /identity/v3/users/52e0587e31d545c5a3ad9ac0a55a8173 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:23.815218 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:23.816857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:23.831795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac666f3-36d4-4e65-a985-b93eda8d5ffd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:23.897528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:23.898234 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:23.898517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:23.898793 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] PATH_INFO: `/v3/projects/5609eb7cbba745a886d59a10ebf92dac` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:23.899956 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5609eb7cbba745a886d59a10ebf92dac)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:23.901950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:23.962144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5609eb7cbba745a886d59a10ebf92dac', 'request_id': 'req-531ca6f6-1a0e-4921-8fd7-99b813c36d51'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.967355 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5609eb7cbba745a886d59a10ebf92dac', 'request_id': 'req-531ca6f6-1a0e-4921-8fd7-99b813c36d51'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.979057 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5609eb7cbba745a886d59a10ebf92dac', 'request_id': 'req-531ca6f6-1a0e-4921-8fd7-99b813c36d51'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.991700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] The token cache is being invalidate because project 5609eb7cbba745a886d59a10ebf92dac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:23.992021 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.992726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-531ca6f6-1a0e-4921-8fd7-99b813c36d51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:23.994336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1319/2645] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:23 2022] DELETE /identity/v3/projects/5609eb7cbba745a886d59a10ebf92dac => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:24.001079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.001701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.001701 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.022309 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:24.023588 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:24.093872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1327/2646] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:24.104390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.104636 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.104891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.126740 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:24.127997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:24.136791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1320/2647] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:24.141362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.151806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eeac-91da-44f2-b5a8-0f7c9781b482 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.157251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.157876 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.158129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.158392 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.159037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:24.159368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.160974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.174177 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1328/2648] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:24.178782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.188284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c8a23-93c4-46e1-a525-ffef589071d2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.193319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.193935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.194209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.194476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.195156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.197093 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.228052 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1321/2649] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:24 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:24.232473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.246058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8751e61-448f-468f-a3d9-b09a63bad52e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.322274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.322903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.323216 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.323550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.324207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.326157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.349532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1329/2650] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:24 2022] POST /identity/v3/users => generated 286 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:24.354489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.365967 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac19580-5f58-4c25-a392-09ae232792c2 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.371880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.372502 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.372781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.373043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.373697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.375368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.388883 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1322/2651] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:24.393696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.406310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f14b4-accd-435f-9817-4a29295e734d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.411860 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.412351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.412753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.413063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.413738 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.415530 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017b9062-25d0-4e72-b856-04b24bc2d712 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.426495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1330/2652] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:24.431344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.449370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db943e1-28da-464b-b570-89b261c898d8 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.455009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.455736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.456015 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.456286 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] PATH_INFO: `/v3/projects/a0577566173748398231bcf8c2d04915/users/57581a73d8e247828987687f17d6416b/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.473765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0577566173748398231bcf8c2d04915, user_id=57581a73d8e247828987687f17d6416b, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.476805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.491183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1323/2653] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:24 2022] PUT /identity/v3/projects/a0577566173748398231bcf8c2d04915/users/57581a73d8e247828987687f17d6416b/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:24.496914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f23d-46e7-436a-920c-508696ae0ba3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.497198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f23d-46e7-436a-920c-508696ae0ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.497470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f44f23d-46e7-436a-920c-508696ae0ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.517514 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5f44f23d-46e7-436a-920c-508696ae0ba3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:24.518748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5f44f23d-46e7-436a-920c-508696ae0ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:24.568270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:24.581054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1331/2654] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:24.595440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:24.597307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:24.629732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:24.659149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3e0fe-9140-4fed-bd3c-64bd61aeb132 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:24.662968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:24.663625 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:24.663897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:24.664172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:24.674322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:24.675853 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:24.685880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1324/2655] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:25.397845 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.398454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.398454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.434362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] MFA Rules not processed for user `57581a73d8e247828987687f17d6416b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:25.435689 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:25.511904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1332/2656] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:25.516729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.527489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb563bf-58bf-4dcd-b195-916306818aab service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.532239 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.532949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.533272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.533546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.534183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.536214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.566785 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1325/2657] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:25 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:25.571298 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.582819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22a8a8-f892-4b41-8232-2ea7d18f8aa8 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.658361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.658980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.659287 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.659581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.660223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.662187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.689586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1333/2658] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:25 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:25.693655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.704874 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11664d96-de8b-4a36-a36f-5cb40bb74ee2 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.709976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.710584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.710891 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.711243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.711896 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.714433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.726657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1326/2659] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:25.731286 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.741352 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e1798-4915-49ae-aa30-8c2278defa8e admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.746060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.746719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.746989 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.747288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] PATH_INFO: `/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf/users/9558044e88094fb59235a6b2e539a501/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.764025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb8f4186e8d42cfb3ed36f4c64a14bf, user_id=9558044e88094fb59235a6b2e539a501, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.766656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.780841 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1334/2660] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:25 2022] PUT /identity/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf/users/9558044e88094fb59235a6b2e539a501/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:25.785644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.798702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9bcb80-6f60-4d79-a138-48ca3fa1dae7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.862162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.862695 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.862976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.863298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.863944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.865593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da2929b-81c1-4038-bb1f-d20eeb98ebc7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.877565 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1327/2661] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:25.886367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:25.899354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628e10ac-6e92-4a55-b257-40d246f7114f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:25.907954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:25.908819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:25.909099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:25.909399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] PATH_INFO: `/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf/users/9558044e88094fb59235a6b2e539a501/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:25.912596 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb8f4186e8d42cfb3ed36f4c64a14bf, user_id=9558044e88094fb59235a6b2e539a501, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:25.916049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:25.931133 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1335/2662] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:25 2022] PUT /identity/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf/users/9558044e88094fb59235a6b2e539a501/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:26.740314 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:26.740958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:26.740958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:26.779178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] MFA Rules not processed for user `9558044e88094fb59235a6b2e539a501`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:26.780606 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:26.879322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1328/2663] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:26.891469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:26.893638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:26.947952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:26.965584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89414a49-ae47-4b70-94b5-65cda0270391 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:26.969260 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:26.970241 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:26.970519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:26.970791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:27.028099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:27.029664 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:27.039955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1336/2664] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:29.450372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:29.452196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:29.475163 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:29.531970 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12e87d7-e176-4794-8022-c3752de65ee0 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:29.538066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:29.538724 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:29.538994 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:29.539303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] PATH_INFO: `/v3/users/d98edbb359184b8f9d4c37533cf1c889` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:29.541716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98edbb359184b8f9d4c37533cf1c889)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:29.542116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:29.543741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:29.543897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:29.556751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cfbdc3-fc8e-47b8-be3c-ba22dd5b9d69 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:29.561907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:29.562608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:29.562958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:29.563391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:29.574079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:29.575801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5af0d0-bfda-4909-9f8b-1b027fa3f787 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:29.587519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1337/2665] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:29.602600 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98edbb359184b8f9d4c37533cf1c889'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:29.609862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98edbb359184b8f9d4c37533cf1c889'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:29.618010 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98edbb359184b8f9d4c37533cf1c889'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:29.618631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98edbb359184b8f9d4c37533cf1c889'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:29.619415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98edbb359184b8f9d4c37533cf1c889'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:29.639271 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1329/2666] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:29 2022] DELETE /identity/v3/users/d98edbb359184b8f9d4c37533cf1c889 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:29.644022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:29.644304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:29.644656 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:29.666473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:29.667996 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:29.736646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1338/2667] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:29.748920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:29.751404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:29.828017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc9a5c-0b83-4f73-a214-499a2b6b5842 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:29.832396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:29.833033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:29.833326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:29.833586 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:29.853503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:29.855949 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:29.872038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1330/2668] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:30.195409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:30.197090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:30.213764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae2120-b681-4f4f-b9be-2ae83678074a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:30.220442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:30.221109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:30.221408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:30.221669 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] PATH_INFO: `/v3/projects/dff1f541b39e436e9f64df23295ce5e1` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:30.223575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dff1f541b39e436e9f64df23295ce5e1)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:30.228785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:30.301507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dff1f541b39e436e9f64df23295ce5e1', 'request_id': 'req-4bdab952-cba8-407a-82dd-d638850705a7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.307620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dff1f541b39e436e9f64df23295ce5e1', 'request_id': 'req-4bdab952-cba8-407a-82dd-d638850705a7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.316168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dff1f541b39e436e9f64df23295ce5e1', 'request_id': 'req-4bdab952-cba8-407a-82dd-d638850705a7'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.329781 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] The token cache is being invalidate because project dff1f541b39e436e9f64df23295ce5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:30.330050 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.330733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.332312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1339/2669] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:30 2022] DELETE /identity/v3/projects/dff1f541b39e436e9f64df23295ce5e1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:30.336430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:30.350987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbfd54-4848-4e3f-a372-5f24ce708d51 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:30.399106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:30.426017 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:30.426691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:30.426979 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:30.427289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] PATH_INFO: `/v3/users/06f874495e8c46c78e5a36741e32bb70` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:30.428583 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f874495e8c46c78e5a36741e32bb70)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:30.430569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:30.494525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f874495e8c46c78e5a36741e32bb70'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.502615 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f874495e8c46c78e5a36741e32bb70'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.513189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f874495e8c46c78e5a36741e32bb70'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.514119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f874495e8c46c78e5a36741e32bb70'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.514735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-78b7483f-dd06-4f8b-8871-44980c2fb5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f874495e8c46c78e5a36741e32bb70'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:30.537238 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1331/2670] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:30 2022] DELETE /identity/v3/users/06f874495e8c46c78e5a36741e32bb70 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:30.841895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:30.843367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:30.857047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:30.864276 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdab952-cba8-407a-82dd-d638850705a7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:30.934623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:30.935495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:30.935839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:30.936240 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] PATH_INFO: `/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:30.937609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9b0a382e6584ca79562dc8baec3fcdd)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:30.939877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:30.997833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9b0a382e6584ca79562dc8baec3fcdd', 'request_id': 'req-967785ab-e32f-445e-a17b-384a91819968'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:31.004394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9b0a382e6584ca79562dc8baec3fcdd', 'request_id': 'req-967785ab-e32f-445e-a17b-384a91819968'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:31.022404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9b0a382e6584ca79562dc8baec3fcdd', 'request_id': 'req-967785ab-e32f-445e-a17b-384a91819968'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:31.035057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] The token cache is being invalidate because project f9b0a382e6584ca79562dc8baec3fcdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:31.035406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:31.036108 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-967785ab-e32f-445e-a17b-384a91819968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:31.037943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1340/2671] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:30 2022] DELETE /identity/v3/projects/f9b0a382e6584ca79562dc8baec3fcdd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:31.043001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.043531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.043884 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.065661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:31.066989 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:31.179042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1332/2672] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:31.187270 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.187560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.187834 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.211310 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:31.212703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:31.229752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1341/2673] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:31.235283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.247319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1fbaa4-6fc9-4b68-9cff-4e86f1faf254 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.252512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.253154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.253433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.253700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.254428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:31.254731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.256416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.269837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1333/2674] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:31.274413 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.284328 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea522a-e293-461f-b490-0440ad90b717 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.289602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.290232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.290507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.290775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.291485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.293474 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.323017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1342/2675] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:31.327561 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.338535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87a62-dd4a-4389-9519-dfa60ceaa16f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.398110 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:31.413547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.414165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.414444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.414719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.415399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.417476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.439836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1334/2676] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:31.445343 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.456741 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1355830f-d6a2-484d-a2cc-61878ef82303 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.462059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.462731 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.463048 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.463402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.464109 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.465792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.477863 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1343/2677] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:31.482376 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.493422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd80079a-21be-4e8f-a2bb-d5a14b31c562 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.499296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.500326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.500326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.500644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.501126 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.502775 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251fff2-6588-4cc8-b289-ae9d6b013dd3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.515372 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1335/2678] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:31.519933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.530783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa822c-c35c-406a-9768-8ab3701e8db1 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.535874 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.536562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.536843 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.537112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] PATH_INFO: `/v3/projects/36200db6b1c8457f972bb2846aaba07a/users/a58e0e1103d3469e812856870fc6e8aa/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.554295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36200db6b1c8457f972bb2846aaba07a, user_id=a58e0e1103d3469e812856870fc6e8aa, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.557100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.573912 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1344/2679] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:31 2022] PUT /identity/v3/projects/36200db6b1c8457f972bb2846aaba07a/users/a58e0e1103d3469e812856870fc6e8aa/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:31.587730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.588223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.588789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.640607 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] MFA Rules not processed for user `a58e0e1103d3469e812856870fc6e8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:31.642760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:31.761024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1336/2680] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:31.768236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.806169 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:31.849252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce48c6-4c25-40ae-8176-74674bbc0b49 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.854592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.855291 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.855580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.855883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.856537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.858532 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:31.898383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1345/2681] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/projects => generated 399 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:31.899910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:31.912469 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd2a1a-c50a-42f3-826f-3956cdfd2fbc None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:31.984214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:31.984844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:31.985124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:31.985405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:31.986062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:31.988086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.013825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1337/2682] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:31 2022] POST /identity/v3/users => generated 306 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:32.017506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.028510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1ec04-bcea-47f3-a193-1cc043f6a8bf admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.033522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.034156 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.034442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.034737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.035433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.037079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.049253 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1346/2683] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:32.054680 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.066354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d1f89-5c35-4b67-9323-fab3e4c2b1ba admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.071253 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.071942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.072221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.072523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] PATH_INFO: `/v3/projects/22b5a8d36b40465fbabec52f34480bec/users/1abddafd3f8544549eb98ea836c0ed85/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.090667 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b5a8d36b40465fbabec52f34480bec, user_id=1abddafd3f8544549eb98ea836c0ed85, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.093455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.111344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1338/2684] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:32 2022] PUT /identity/v3/projects/22b5a8d36b40465fbabec52f34480bec/users/1abddafd3f8544549eb98ea836c0ed85/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:32.115839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.127806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab0ca35-9a01-470b-8125-d7bcdb535538 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.191320 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.191939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.192214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.192481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.193141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.194750 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321b9ca2-bd9a-4296-b963-fe0c3d666d85 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.205965 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1347/2685] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:32.210148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.222707 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2995b0-13cc-4e72-a265-ea7a7d2cf3be admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.228604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.229426 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.229716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.229998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] PATH_INFO: `/v3/projects/22b5a8d36b40465fbabec52f34480bec/users/1abddafd3f8544549eb98ea836c0ed85/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.232411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22b5a8d36b40465fbabec52f34480bec, user_id=1abddafd3f8544549eb98ea836c0ed85, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.235179 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.252201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1339/2686] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:32 2022] PUT /identity/v3/projects/22b5a8d36b40465fbabec52f34480bec/users/1abddafd3f8544549eb98ea836c0ed85/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:32.265621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.266006 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.266444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.328801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] MFA Rules not processed for user `1abddafd3f8544549eb98ea836c0ed85`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:32.331043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:32.428769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1348/2687] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:32 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:32.440714 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.443006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:32.467197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:32.542551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2694deb1-eeef-4025-9d1b-8938bbc275f5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.547295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.548120 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.548566 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.549012 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.561582 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.563991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.577769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1340/2688] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:32.673569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:32.685026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f226ff5b-3924-40dd-b6e8-7eb03fd69a8d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:32.689454 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:32.690088 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:32.690363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:32.690628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:32.716122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:32.767204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:32.768599 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:32.780010 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1349/2689] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:37.035429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:37.037423 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:37.079571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:37.112834 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e65a5-15fa-4a8b-8343-31ab24711813 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:37.116892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:37.117533 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:37.117808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:37.118076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:37.120953 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:37.164733 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0e2a7-a4f2-4ab3-98e7-7ea786697ffe service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:37.170037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:37.171042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:37.171438 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:37.171849 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] PATH_INFO: `/v3/users/a58e0e1103d3469e812856870fc6e8aa` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:37.173447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58e0e1103d3469e812856870fc6e8aa)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:37.176355 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:37.224245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:37.225749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd988baa-3355-4e82-b198-d6831b4c7298 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:37.240174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1341/2690] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:37.243812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58e0e1103d3469e812856870fc6e8aa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:37.256885 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58e0e1103d3469e812856870fc6e8aa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:37.257783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58e0e1103d3469e812856870fc6e8aa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:37.258442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58e0e1103d3469e812856870fc6e8aa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:37.273809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58e0e1103d3469e812856870fc6e8aa'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:37.302470 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1350/2691] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:37 2022] DELETE /identity/v3/users/a58e0e1103d3469e812856870fc6e8aa => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:37.307659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:37.308293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:37.308811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:37.335997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:37.337728 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:37.433907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:37.450366 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1342/2692] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:37.465361 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:37.467991 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:37.557187 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:37.581405 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a0a098-b3aa-403f-8555-076f575237d9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:37.589373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:37.590021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:37.590277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:37.590549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:37.602087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:37.603636 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:37.614523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1351/2693] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:38.079656 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:38.081211 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:38.112173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c200ed-f587-4d74-8493-423b1a7b4a2b None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:38.118973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:38.120686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:38.121825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:38.122284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] PATH_INFO: `/v3/projects/36200db6b1c8457f972bb2846aaba07a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:38.124662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=36200db6b1c8457f972bb2846aaba07a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:38.127156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:38.196129 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36200db6b1c8457f972bb2846aaba07a', 'request_id': 'req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.206906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36200db6b1c8457f972bb2846aaba07a', 'request_id': 'req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.217761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36200db6b1c8457f972bb2846aaba07a', 'request_id': 'req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.232723 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] The token cache is being invalidate because project 36200db6b1c8457f972bb2846aaba07a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:38.233157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.233955 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.236026 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1343/2694] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:38 2022] DELETE /identity/v3/projects/36200db6b1c8457f972bb2846aaba07a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:38.241711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:38.255003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb677940-dd20-4d5d-9f0a-a98dcbf91e22 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:38.370575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:38.377547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:38.377547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:38.377547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] PATH_INFO: `/v3/users/1abddafd3f8544549eb98ea836c0ed85` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:38.379026 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1abddafd3f8544549eb98ea836c0ed85)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:38.383375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:38.481882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1abddafd3f8544549eb98ea836c0ed85'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.501386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1abddafd3f8544549eb98ea836c0ed85'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.502613 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1abddafd3f8544549eb98ea836c0ed85'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.504865 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1abddafd3f8544549eb98ea836c0ed85'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.524158 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ff873f26-4b35-4cbd-9784-cd6facf3d811 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1abddafd3f8544549eb98ea836c0ed85'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:38.557451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1352/2695] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:38 2022] DELETE /identity/v3/users/1abddafd3f8544549eb98ea836c0ed85 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:39.190905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:39.192826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:39.212049 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b685fc7-1b13-4510-a3b5-3dd8d561b03f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:39.307727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:39.308640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.309027 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.309401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] PATH_INFO: `/v3/projects/22b5a8d36b40465fbabec52f34480bec` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:39.311201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22b5a8d36b40465fbabec52f34480bec)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:39.314196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:39.400597 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22b5a8d36b40465fbabec52f34480bec', 'request_id': 'req-b861c7e4-e5f0-4568-bffa-7232256732c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:39.413477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22b5a8d36b40465fbabec52f34480bec', 'request_id': 'req-b861c7e4-e5f0-4568-bffa-7232256732c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:39.427207 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22b5a8d36b40465fbabec52f34480bec', 'request_id': 'req-b861c7e4-e5f0-4568-bffa-7232256732c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:39.443431 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] The token cache is being invalidate because project 22b5a8d36b40465fbabec52f34480bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:39.444085 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:39.445249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-b861c7e4-e5f0-4568-bffa-7232256732c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:39.447544 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1344/2696] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:39 2022] DELETE /identity/v3/projects/22b5a8d36b40465fbabec52f34480bec => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:39.453182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.453788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.453994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:39.482022 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:39.483868 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:39.613494 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1353/2697] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:39.628882 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.629325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.629818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:39.657980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:39.659412 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:39.676342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1345/2698] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:39.677078 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:39.683888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:39.691653 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62debec4-d6f2-46c9-afdf-01fadaaf828c None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:39.701151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:39.702204 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.703281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.704086 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:39.705423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:39.705907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:39.708491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:39.723670 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1354/2699] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:39.729203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:39.745691 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2347cd61-7d1c-4f13-b406-a29cc6c6b4e7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:39.753425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:39.754436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.754928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.755311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:39.756038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:39.762547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:39.775559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:39.831308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1346/2700] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:39 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:39.835943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:39.854066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:39.857393 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:39.862153 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fa297-2005-4feb-9730-9c9d59778ad1 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:39.983170 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d44bf32-35a2-425b-8401-a644a5951397 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:39.994762 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:39.996803 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:39.997510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:39.998916 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.002480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:40.004648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.006151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.006635 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.007041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.008255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.012447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.022630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.024833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.061710 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1355/2701] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:39 2022] POST /identity/v3/users => generated 292 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:40.062064 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1347/2702] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:46:40.066114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.101928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86495611-f8ae-4324-bf7f-8f9e7861c25a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.112712 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.113711 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.114211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.114614 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.115799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.119608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.146090 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1356/2703] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:40.148145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.170729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b309c-5b18-4886-84b5-94f812faf648 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.177852 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.178595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.178892 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.179201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.179977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.182945 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e3ec1-2586-40cd-b4a9-ce20c5c13d8f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.194760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1348/2704] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:40.199777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.219029 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2276f23-68b7-4ed3-92d8-9c67186be0b3 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.225211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.225936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.226211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.226807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] PATH_INFO: `/v3/projects/741de4250c6b49e196afe60418c43d1e/users/9e48b2315a63402ab7120a35e3d03560/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.254401 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=741de4250c6b49e196afe60418c43d1e, user_id=9e48b2315a63402ab7120a35e3d03560, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.257349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.286762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1357/2705] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:40 2022] PUT /identity/v3/projects/741de4250c6b49e196afe60418c43d1e/users/9e48b2315a63402ab7120a35e3d03560/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:40.300648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.301094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.301556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.356795 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] MFA Rules not processed for user `9e48b2315a63402ab7120a35e3d03560`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:40.359209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:40.469140 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1349/2706] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:40.474430 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.602193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f024a8a-194e-4291-8e2f-e84829197787 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.608290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.608931 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.609209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.609479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.610144 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.612175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.651721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1358/2707] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:40 2022] POST /identity/v3/projects => generated 381 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:40.657749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.690476 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ea259-8ceb-455a-9760-d026abf4fe87 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.835730 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:40.865845 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.868031 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.868459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.868996 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.870416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.875127 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.929151 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1350/2708] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:40 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:40.933042 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:40.950117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3446f-41ee-42ca-a760-f54dff25191a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:40.958963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:40.959850 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:40.960186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:40.960504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:40.961180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:40.964119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:40.979138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1359/2709] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:40.985006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:41.013440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe61a000-b063-4afc-84cf-60cb947214e6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:41.023569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:41.024732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.025230 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.025721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] PATH_INFO: `/v3/projects/ac3aea4dcda04ef48571c2487c2e0597/users/17e3229cd91246fd9400e49e6d534b7a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:41.061444 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3aea4dcda04ef48571c2487c2e0597, user_id=17e3229cd91246fd9400e49e6d534b7a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:41.071549 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:41.120273 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1351/2710] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:40 2022] PUT /identity/v3/projects/ac3aea4dcda04ef48571c2487c2e0597/users/17e3229cd91246fd9400e49e6d534b7a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:41.129136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:41.156603 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2122438c-f136-4fbb-83c7-8c8605ef93f7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:41.202869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:41.284112 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:41.284764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.285065 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.285396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:41.286080 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:41.291409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4654db6b-45e1-4296-a71e-94b06fcdae08 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:41.315249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1360/2711] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:41 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:41.319681 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:41.331555 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f983c3-d69a-421e-a1b1-2ed196d88a6b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:41.338223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:41.339065 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.339575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.339856 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] PATH_INFO: `/v3/projects/ac3aea4dcda04ef48571c2487c2e0597/users/17e3229cd91246fd9400e49e6d534b7a/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:41.342265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3aea4dcda04ef48571c2487c2e0597, user_id=17e3229cd91246fd9400e49e6d534b7a, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:41.346593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:41.367808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1352/2712] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:41 2022] PUT /identity/v3/projects/ac3aea4dcda04ef48571c2487c2e0597/users/17e3229cd91246fd9400e49e6d534b7a/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:41.399238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.399777 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.400427 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:41.441783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] MFA Rules not processed for user `17e3229cd91246fd9400e49e6d534b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:41.443072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:41.546389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1361/2713] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:41.561102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:41.563639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:41.674534 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68febce-fedc-4f13-acf5-82f81e22b19d admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:41.679428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:41.680167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.680514 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.680859 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:41.703156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:41.706602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:41.728640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1353/2714] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:46:41.912368 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:41.914909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:41.934465 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00b51e-9584-4b06-bc76-442e32750586 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:41.940121 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:41.941205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:41.941727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:41.942176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:42.035649 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:42.037152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:42.050268 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1362/2715] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:44.794973 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:44.799799 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:44.892899 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7d090-1d20-4175-b11a-4bbf6d429283 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:44.899911 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:44.900929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:44.901370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:44.901857 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] PATH_INFO: `/v3/users/57581a73d8e247828987687f17d6416b` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:44.903919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=57581a73d8e247828987687f17d6416b)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:44.907242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:44.935916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:44.952520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb656bf2-518f-44b6-a872-a5f3a0fd6036 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:44.959277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:44.960151 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:44.960619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:44.961018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] PATH_INFO: `/v3/users/9e48b2315a63402ab7120a35e3d03560` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:44.963176 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e48b2315a63402ab7120a35e3d03560)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:44.965934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:44.974156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:44.976266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57581a73d8e247828987687f17d6416b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:44.986083 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57581a73d8e247828987687f17d6416b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:44.994640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57581a73d8e247828987687f17d6416b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:44.995414 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57581a73d8e247828987687f17d6416b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:44.996100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-16e6179c-4a12-402d-889c-dcbc6e905d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57581a73d8e247828987687f17d6416b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.019300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1354/2716] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:44 2022] DELETE /identity/v3/users/57581a73d8e247828987687f17d6416b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:45.027336 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:45.040561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e48b2315a63402ab7120a35e3d03560'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.050541 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e48b2315a63402ab7120a35e3d03560'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.051572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e48b2315a63402ab7120a35e3d03560'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.052452 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e48b2315a63402ab7120a35e3d03560'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.063721 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e48b2315a63402ab7120a35e3d03560'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.087577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1363/2717] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:44 2022] DELETE /identity/v3/users/9e48b2315a63402ab7120a35e3d03560 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:45.091699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.092054 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.092336 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.118944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:45.121388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.208729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1355/2718] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:45.219301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.221338 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.271182 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.273331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.286796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f59c0-40d0-4e08-99e9-aeb5a83f3222 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:45.292735 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:45.293385 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.293679 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.294048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] PATH_INFO: `/v3/projects/a0577566173748398231bcf8c2d04915` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.295380 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0577566173748398231bcf8c2d04915)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:45.297623 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:45.331138 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4840865-f116-40fb-957e-761e65ab24be admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:45.337696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:45.338651 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.339008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.339440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.351410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:45.352870 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:45.362719 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0577566173748398231bcf8c2d04915', 'request_id': 'req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.368748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1364/2719] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:46:45.370880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0577566173748398231bcf8c2d04915', 'request_id': 'req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.384671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0577566173748398231bcf8c2d04915', 'request_id': 'req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.399115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] The token cache is being invalidate because project a0577566173748398231bcf8c2d04915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:45.399419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.400240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.402256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1356/2720] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:45 2022] DELETE /identity/v3/projects/a0577566173748398231bcf8c2d04915 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:45.406129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.416886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebce729-0e83-496e-815f-6456b5aa1b52 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:45.485165 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:45.485812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.487506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.487506 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] PATH_INFO: `/v3/users/9558044e88094fb59235a6b2e539a501` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.487815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9558044e88094fb59235a6b2e539a501)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:45.489825 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:45.557168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9558044e88094fb59235a6b2e539a501'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.567495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9558044e88094fb59235a6b2e539a501'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.568372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9558044e88094fb59235a6b2e539a501'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.568953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9558044e88094fb59235a6b2e539a501'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.572496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:45.582697 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9558044e88094fb59235a6b2e539a501'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.609018 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1365/2721] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:45 2022] DELETE /identity/v3/users/9558044e88094fb59235a6b2e539a501 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:45.744751 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.746396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.769131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddc6d3-6b42-4f40-a3bd-14c9b836822f admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:45.831249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:45.853756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:45.854801 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.855114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.855381 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] PATH_INFO: `/v3/projects/741de4250c6b49e196afe60418c43d1e` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.856546 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=741de4250c6b49e196afe60418c43d1e)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:45.858547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:45.883602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.885681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.900409 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee02bdc8-e8e6-41fc-b98d-2bbeb30519b3 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:45.907028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:45.907807 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:45.908174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:45.908471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] PATH_INFO: `/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:45.909837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb8f4186e8d42cfb3ed36f4c64a14bf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:45.911856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:45.932556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '741de4250c6b49e196afe60418c43d1e', 'request_id': 'req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.938251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '741de4250c6b49e196afe60418c43d1e', 'request_id': 'req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.948731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '741de4250c6b49e196afe60418c43d1e', 'request_id': 'req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.960662 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] The token cache is being invalidate because project 741de4250c6b49e196afe60418c43d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:45.960950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.961721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.963467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1357/2722] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:45 2022] DELETE /identity/v3/projects/741de4250c6b49e196afe60418c43d1e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:45.967589 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:45.967761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb8f4186e8d42cfb3ed36f4c64a14bf', 'request_id': 'req-90d0577f-c0ee-4d07-8328-d2e86c86aaad'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.968977 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:45.974775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb8f4186e8d42cfb3ed36f4c64a14bf', 'request_id': 'req-90d0577f-c0ee-4d07-8328-d2e86c86aaad'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.984459 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb8f4186e8d42cfb3ed36f4c64a14bf', 'request_id': 'req-90d0577f-c0ee-4d07-8328-d2e86c86aaad'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:45.998542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] The token cache is being invalidate because project 1fb8f4186e8d42cfb3ed36f4c64a14bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:45.999261 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.000063 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-90d0577f-c0ee-4d07-8328-d2e86c86aaad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.002937 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1366/2723] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:45 2022] DELETE /identity/v3/projects/1fb8f4186e8d42cfb3ed36f4c64a14bf => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:46.010792 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6230a-0187-4013-b782-f117786fc845 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.011378 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6230a-0187-4013-b782-f117786fc845 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.011820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb6230a-0187-4013-b782-f117786fc845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.037608 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-5fb6230a-0187-4013-b782-f117786fc845 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:46.038830 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5fb6230a-0187-4013-b782-f117786fc845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.047192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fda6d-e2b8-473f-97b8-7b77dfe25584 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.126683 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1367/2724] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:46.127771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.128437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.128716 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.128988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] PATH_INFO: `/v3/users/17e3229cd91246fd9400e49e6d534b7a` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.130160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17e3229cd91246fd9400e49e6d534b7a)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.132193 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.134337 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.134625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.134910 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.155691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:46.157299 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.166471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:46.167577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1368/2725] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:46.172372 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.183713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd862b3-b447-4213-a9fb-37f05dc0f599 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.188529 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.189142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.189440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.189705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.190373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:46.190667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.192353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.197337 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17e3229cd91246fd9400e49e6d534b7a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.205930 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17e3229cd91246fd9400e49e6d534b7a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.207346 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1369/2726] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:46.211819 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.215901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17e3229cd91246fd9400e49e6d534b7a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.216672 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17e3229cd91246fd9400e49e6d534b7a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.217447 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17e3229cd91246fd9400e49e6d534b7a'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.230069 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee8fab-8b9e-425b-904b-d72c43692bcd admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.235969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.236688 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.237179 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.237518 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.238226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.240394 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.241854 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1358/2727] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:45 2022] DELETE /identity/v3/users/17e3229cd91246fd9400e49e6d534b7a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:46.279124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1370/2728] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:46.282516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.284133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.367018 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e7aeb-e36b-4d7c-a4a8-468b81973b74 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.451671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.452300 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.452490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.452757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.453388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.455388 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.479362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1359/2729] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:46.484579 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.495543 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720c7a1-c68c-4b1f-8e8d-a20c452e1ff0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.500728 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.501354 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.501633 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.501901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.502552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.504197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.515386 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1371/2730] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:46.519700 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.530815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c823051a-506d-4fc1-b803-5f09d7158b6a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.535668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.536289 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.536593 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.536863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.537518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.539187 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861717f-e3f5-43ea-ba6c-d4e11b928858 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.549900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1360/2731] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:46.554266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.561730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:46.566414 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35af10-6a99-43db-bb5a-bc8e5e50c44c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.572808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.574278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.574278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.574278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] PATH_INFO: `/v3/projects/6e1c4ab2621a41a0aafc6679e877d9b9/users/b89323e79b274434b6826e1189211035/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.592917 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e1c4ab2621a41a0aafc6679e877d9b9, user_id=b89323e79b274434b6826e1189211035, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.595982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.620457 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1372/2732] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:46 2022] PUT /identity/v3/projects/6e1c4ab2621a41a0aafc6679e877d9b9/users/b89323e79b274434b6826e1189211035/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:46.631490 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.631976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.632395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.634556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.637101 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.682798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] MFA Rules not processed for user `b89323e79b274434b6826e1189211035`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:46.684067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.702595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:46.717793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8e5f36-a6f8-4eb4-b1cc-7802d7ae0c2a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.723237 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.723895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.724195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.724549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] PATH_INFO: `/v3/projects/ac3aea4dcda04ef48571c2487c2e0597` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.725631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3aea4dcda04ef48571c2487c2e0597)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.727577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.762317 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1361/2733] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:46.767371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.784495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d0610-3d52-4ec5-9c70-393f7920bdca None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.799225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3aea4dcda04ef48571c2487c2e0597', 'request_id': 'req-bfcff798-4245-41b6-9d6f-b570d7615c30'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.805205 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3aea4dcda04ef48571c2487c2e0597', 'request_id': 'req-bfcff798-4245-41b6-9d6f-b570d7615c30'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.815451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3aea4dcda04ef48571c2487c2e0597', 'request_id': 'req-bfcff798-4245-41b6-9d6f-b570d7615c30'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.830528 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] The token cache is being invalidate because project ac3aea4dcda04ef48571c2487c2e0597 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:46.830833 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.831581 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-bfcff798-4245-41b6-9d6f-b570d7615c30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:46.833203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1373/2734] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:46 2022] DELETE /identity/v3/projects/ac3aea4dcda04ef48571c2487c2e0597 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:46.839942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac024e-f68a-47c9-b2a2-6ab4ef2bc009 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.840234 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac024e-f68a-47c9-b2a2-6ab4ef2bc009 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.840509 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac024e-f68a-47c9-b2a2-6ab4ef2bc009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.861620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-48ac024e-f68a-47c9-b2a2-6ab4ef2bc009 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:46.863043 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-48ac024e-f68a-47c9-b2a2-6ab4ef2bc009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.866609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:46.867755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.868269 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.868577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.869235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:46.871492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:46.871700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1374/2735] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:46.880519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:46.880797 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:46.881077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:46.902974 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:46.904626 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.915304 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1362/2736] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:46.916387 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1375/2737] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:46.916564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.918130 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:46.921347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:46.932828 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f7c8a-2772-4fb1-a77d-73a093eb4a7d None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.993283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d71653-6273-4150-8b7c-323f7c85e90c admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:46.997238 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:47.004457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.004774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.005104 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.005402 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.005538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.005671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.005775 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.005976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.006350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.007135 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:47.007817 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.008325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.009652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.030765 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1376/2738] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:46:47.032347 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.035148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1363/2739] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:46 2022] POST /identity/v3/users => generated 299 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:47.039203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.044358 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de9f59-b734-4a75-bfc9-dc7ba7e0d1da admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.051401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6d5ce-c894-4b98-a0f7-217ac37f76e1 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.052726 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.053441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.053751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.054083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.055351 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.056571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.057195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.057477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.057813 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.058307 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.058474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.060131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.072609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1364/2740] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:47.077109 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.088303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5fae76-7f93-434a-b265-63f997f29da6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.092059 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1377/2741] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:47 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:46:47.096770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.097062 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.097823 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.098178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.098512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] PATH_INFO: `/v3/projects/1f2fea620027499ca987b422549e06a9/users/7a94822b0f6b44b1b2673b62ccdd6920/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.116591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2fea620027499ca987b422549e06a9, user_id=7a94822b0f6b44b1b2673b62ccdd6920, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.119298 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.130242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:47.136889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1365/2742] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:47 2022] PUT /identity/v3/projects/1f2fea620027499ca987b422549e06a9/users/7a94822b0f6b44b1b2673b62ccdd6920/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:47.141157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.182185 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159bf65b-cdbc-4a61-b540-0d77ca14d07a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.218641 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e6e46-582e-429b-9721-9a14702325c2 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.224133 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.224765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.225039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.225311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.226040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.227763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.239034 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1366/2743] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:47.245418 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.260102 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.260883 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.261233 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.261607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.262285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fed43a2-8b30-4bf2-86d5-22b882e68a04 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.262403 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.264418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.268006 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.268798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.268798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.268798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] PATH_INFO: `/v3/projects/1f2fea620027499ca987b422549e06a9/users/7a94822b0f6b44b1b2673b62ccdd6920/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.273400 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2fea620027499ca987b422549e06a9, user_id=7a94822b0f6b44b1b2673b62ccdd6920, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.278321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c118c297-0df0-4d7c-b61c-99ceed9d7ad1 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.295376 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1378/2744] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:47 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:47.297668 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.298156 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1367/2745] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:47 2022] PUT /identity/v3/projects/1f2fea620027499ca987b422549e06a9/users/7a94822b0f6b44b1b2673b62ccdd6920/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:47.315185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.315185 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.315489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.317691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb53d50-18bd-4c19-acf7-c3dbe75f70c9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.350310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] MFA Rules not processed for user `7a94822b0f6b44b1b2673b62ccdd6920`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:47.351590 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:47.396772 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.397395 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.397675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.397952 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.398612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.400274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.414943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1379/2746] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:47.419033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.424407 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:47.430861 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d8313-69f1-4ff2-8052-9c05a9224dab admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.436950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.437585 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.437857 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.438132 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.438285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1368/2747] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:47.438787 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.440491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.451529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.452456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1380/2748] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:47.454094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:47.456676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.467275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2c5a0-78e4-4023-9246-ac923ae4f10d admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.473592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.474278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.474550 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.474827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] PATH_INFO: `/v3/projects/f4d965940a72402b8706a0967ea1a243/users/9650e14e8dfb4ed3a16498fdfb443a49/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.491844 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4d965940a72402b8706a0967ea1a243, user_id=9650e14e8dfb4ed3a16498fdfb443a49, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.494561 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc400f3-73aa-4909-a1e0-6b3a16b282b6 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.509591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1381/2749] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:47 2022] PUT /identity/v3/projects/f4d965940a72402b8706a0967ea1a243/users/9650e14e8dfb4ed3a16498fdfb443a49/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 08 13:46:47.520304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.520589 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.520873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.543256 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4229544-2df2-4ae3-9349-d7f0d860d870 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.562672 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] MFA Rules not processed for user `9650e14e8dfb4ed3a16498fdfb443a49`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:47.563326 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:47.564663 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:47.601539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.602165 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.602443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.602717 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.605815 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:47.644007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1382/2750] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:46:47.658313 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:47.659960 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:47.677368 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.678867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.689963 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1369/2751] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:47.734297 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06693f-1762-4954-984e-d4c158a4919a None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:47.738806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f034fd79c9ef454498ee0874d2c97d16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:47.739515 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:47.739795 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:47.740062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:47.749508 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:47.751071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:47.762326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1383/2752] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:46:52.207351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:52.221429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61423556-30fc-43a6-a2ed-052809e07896 service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:52.226550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:52.227241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:52.227522 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:52.227791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:52.305749 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:52.307209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231695c5-9a30-4fe9-a81f-d4f74a64ea2c service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:52.321563 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1370/2753] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:57.388736 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:57.465874 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2895e47-a30a-41ca-8072-775dd45c1eee service glance] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:57.471820 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:57.472507 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:57.472958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:57.473236 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] PATH_INFO: `/v3/users/b89323e79b274434b6826e1189211035` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:57.474446 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b89323e79b274434b6826e1189211035)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:57.476469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:57.478737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:57.530891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b89323e79b274434b6826e1189211035'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:57.539175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b89323e79b274434b6826e1189211035'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:57.539985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b89323e79b274434b6826e1189211035'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:57.540641 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b89323e79b274434b6826e1189211035'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:57.550263 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b89323e79b274434b6826e1189211035'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:57.572455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1384/2754] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:57 2022] DELETE /identity/v3/users/b89323e79b274434b6826e1189211035 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:57.577991 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:57.578748 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:57.579236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:57.603471 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:57.604742 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:57.666144 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1371/2755] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:57.677242 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:57.678835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:57.758068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911c5d-a066-41f9-b295-01d1b255a49c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:57.764117 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:57.764745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:57.765105 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:57.765362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:57.775537 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:57.777028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:57.788224 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1385/2756] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:46:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:46:58.101548 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:58.103280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:58.116798 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf4abb1-28c8-4e7a-a43b-5a1da106d413 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:58.123360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:58.124020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:58.124297 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:58.124552 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] PATH_INFO: `/v3/projects/6e1c4ab2621a41a0aafc6679e877d9b9` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:58.125737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e1c4ab2621a41a0aafc6679e877d9b9)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:58.127805 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:58.135455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:58.187158 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e1c4ab2621a41a0aafc6679e877d9b9', 'request_id': 'req-dc2872db-a51d-4567-a9e4-cfcd2de71f88'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.193501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e1c4ab2621a41a0aafc6679e877d9b9', 'request_id': 'req-dc2872db-a51d-4567-a9e4-cfcd2de71f88'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.202907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e1c4ab2621a41a0aafc6679e877d9b9', 'request_id': 'req-dc2872db-a51d-4567-a9e4-cfcd2de71f88'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.216512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] The token cache is being invalidate because project 6e1c4ab2621a41a0aafc6679e877d9b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:58.216862 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.217578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.219340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1372/2757] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:58 2022] DELETE /identity/v3/projects/6e1c4ab2621a41a0aafc6679e877d9b9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:58.224251 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:58.234907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1d6186-8e14-4cdc-bbdd-85cccf541517 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:58.311032 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:58.311757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:58.312033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:58.312303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] PATH_INFO: `/v3/users/7a94822b0f6b44b1b2673b62ccdd6920` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:58.313571 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a94822b0f6b44b1b2673b62ccdd6920)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:58.315560 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:58.342472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:58.372873 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a94822b0f6b44b1b2673b62ccdd6920'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.381230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a94822b0f6b44b1b2673b62ccdd6920'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.382140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a94822b0f6b44b1b2673b62ccdd6920'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.382793 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a94822b0f6b44b1b2673b62ccdd6920'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.395182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-8539e010-acf8-4f71-a958-2de6757db082 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a94822b0f6b44b1b2673b62ccdd6920'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.414729 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1386/2758] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:46:58 2022] DELETE /identity/v3/users/7a94822b0f6b44b1b2673b62ccdd6920 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:58.814432 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:58.816266 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:58.831844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2872db-a51d-4567-a9e4-cfcd2de71f88 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:58.903360 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:58.904032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:58.904313 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:58.904599 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] PATH_INFO: `/v3/projects/1f2fea620027499ca987b422549e06a9` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:58.905817 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2fea620027499ca987b422549e06a9)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:58.907789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:58.968704 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2fea620027499ca987b422549e06a9', 'request_id': 'req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.974539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2fea620027499ca987b422549e06a9', 'request_id': 'req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.981897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2fea620027499ca987b422549e06a9', 'request_id': 'req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.993542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] The token cache is being invalidate because project 1f2fea620027499ca987b422549e06a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:46:58.993831 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.994542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-9b4eb22b-e14a-4690-9b49-0eefcad0aedd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:46:58.996250 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1373/2759] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:46:58 2022] DELETE /identity/v3/projects/1f2fea620027499ca987b422549e06a9 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:59.002362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.002650 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.002929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.023106 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:59.024396 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:59.088433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1387/2760] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:59.098356 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.098638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.098910 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.129350 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:59.130564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:59.142833 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1374/2761] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:59.147922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.159257 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f3577-e098-4af5-b4b4-0e0716e59a88 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.164505 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.165183 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.165440 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.165706 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.166362 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:46:59.166674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.168348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.180574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1388/2762] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:46:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:46:59.185480 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.195575 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8337b71-285a-4ea2-b982-646c76bd0030 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.201039 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.201648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.201928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.202203 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.202836 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.204901 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.221720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:59.238559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1375/2763] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:59.242895 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.253601 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66aef0e-c3b7-4d97-8a06-5d4b3774846a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.314120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.314800 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.315211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.315485 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.316126 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.318074 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.339654 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1389/2764] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:59.344186 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.355602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5ca37-5ab6-483a-98d7-64e689a36cc4 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.361001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.361604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.361897 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.362152 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.362789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.364446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.375024 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1376/2765] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:59.379444 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.390033 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f227735a-9cdf-459d-baaf-9dcd1e5bb14f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.395744 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.396410 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.396637 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.396933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.397584 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.399229 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bae62d-4245-40d7-979d-cdba3ee3f860 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.416028 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1390/2766] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:59.420448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.431287 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9af345-8ebe-4ede-988e-5a8587d0d0d5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.437096 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.437763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.438036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.438332 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] PATH_INFO: `/v3/projects/80a33c7355ec451c8ac20f4b187380b6/users/da1310515be84efaa2fc3b0ee19f5044/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.458307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a33c7355ec451c8ac20f4b187380b6, user_id=da1310515be84efaa2fc3b0ee19f5044, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.461255 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.476574 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1377/2767] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:59 2022] PUT /identity/v3/projects/80a33c7355ec451c8ac20f4b187380b6/users/da1310515be84efaa2fc3b0ee19f5044/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:46:59.488130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.488428 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.488707 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.503318 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:46:59.521835 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] MFA Rules not processed for user `da1310515be84efaa2fc3b0ee19f5044`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:46:59.523111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:46:59.596312 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1391/2768] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:46:59.601806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.666657 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1212b1-6cf8-4d32-89d8-e805ad14b45e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.671674 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.672281 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.672559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.672826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.673451 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.675399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.707638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1378/2769] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:59.713493 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.726970 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03395bc-1de5-48b0-a85e-0817329fde80 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.786901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.787549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.787827 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.788097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.788732 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.790667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.812034 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1392/2770] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:46:59 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:46:59.816889 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.832611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa24a95-5f4b-488f-a1b3-c901f8eb63f6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.840378 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.841101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.841404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.841676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.842324 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.844032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.855712 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1379/2771] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:46:59.859921 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.871036 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc50dab-a503-4448-82b9-993ccc095d55 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.876323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.876934 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.877211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.877478 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.878111 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.879769 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0372b6-c16d-4c9f-9f70-a4effffe8de1 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.890555 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1393/2772] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:46:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:46:59.895009 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:46:59.906411 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ccc50f-9ce8-442a-a3b9-9daa6a610d11 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:46:59.911863 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:46:59.912542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:46:59.912824 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:46:59.913098 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] PATH_INFO: `/v3/projects/a438d405c5d241988dfdba2c22c55116/users/dc10d5bf37804de2aa47887cc54bb965/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:46:59.929322 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a438d405c5d241988dfdba2c22c55116, user_id=dc10d5bf37804de2aa47887cc54bb965, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:46:59.931936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:46:59.946660 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1380/2773] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:46:59 2022] PUT /identity/v3/projects/a438d405c5d241988dfdba2c22c55116/users/dc10d5bf37804de2aa47887cc54bb965/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:00.018356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.018694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.019041 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.056040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] MFA Rules not processed for user `dc10d5bf37804de2aa47887cc54bb965`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:00.057725 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:00.129655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1394/2774] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:00.136051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.202482 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054dd3c-3a2e-4e34-8f15-d87d8749d2fd admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.207307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.207917 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.208195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.208467 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.209090 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.211011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.226926 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:00.241550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1381/2775] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:47:00 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:00.245666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.253053 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:00.259442 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f47c7e-414e-41dc-93fc-c184dfa747d5 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.327823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.328737 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.329141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.329496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.330495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.333116 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.359527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1395/2776] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:47:00 2022] POST /identity/v3/users => generated 299 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:00.365166 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.378352 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6651e55f-cb35-4930-9d1a-25c31a3e2c82 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.384245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.384866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.385149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.385421 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.386539 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.387780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.402988 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1382/2777] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:47:00.404964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.415510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6079d8-f538-452d-9f17-fda5a285ce0a admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.420472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.421122 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.421397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.421696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] PATH_INFO: `/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25/users/2e746bc7966c403c9d75bc34021841c9/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.440423 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e083c6bdb54f3d9e2445ccb4dc1b25, user_id=2e746bc7966c403c9d75bc34021841c9, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.443100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.458269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1396/2778] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:47:00 2022] PUT /identity/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25/users/2e746bc7966c403c9d75bc34021841c9/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:00.462354 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.472958 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578fd84-67b3-4c85-bc1e-ad2a1d64978e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.536961 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.537595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.537905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.538145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.538807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.540497 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e640ced-d7ed-4a74-8e39-693aea6350e4 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.551780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1383/2779] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:00 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:47:00.557551 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.568886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de7316e-083d-42c6-8c43-edf52da40c13 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.573690 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.574367 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.574646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.574924 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] PATH_INFO: `/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25/users/2e746bc7966c403c9d75bc34021841c9/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.576928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=75e083c6bdb54f3d9e2445ccb4dc1b25, user_id=2e746bc7966c403c9d75bc34021841c9, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.579686 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.590933 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1397/2780] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:47:00 2022] PUT /identity/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25/users/2e746bc7966c403c9d75bc34021841c9/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:00.602676 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.602940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.603280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.634807 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] MFA Rules not processed for user `2e746bc7966c403c9d75bc34021841c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:00.636063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:00.711210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1384/2781] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:00 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:47:00.723547 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:00.725422 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:00.787504 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0fe519-7565-4409-9bc4-dfb16084bcfe admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:00.791735 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:00.792408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:00.792661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:00.792966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:00.851901 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:00.853344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:00.863120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1398/2782] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:00 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:02.429424 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:02.440764 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da46e9-149f-44e2-b507-239ebd45f374 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:02.444487 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:02.445094 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:02.445367 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:02.445632 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:02.510123 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:02.511732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:02.521280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1385/2783] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:02 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:03.154903 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:03.156502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:03.168330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862418a6-4b67-4265-a5ec-d88e1ff174f5 service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:03.172575 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:03.173267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:03.173538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:03.173803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:03.184479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:03.186021 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be856dc-fb15-4c15-aece-fad8d15d433c service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:03.190301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:03.199258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1399/2784] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:08.200308 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:08.206004 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=100 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:08.276551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6e590-6703-4fb7-b8e9-767172037ace service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:08.282950 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:08.283903 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:08.284201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:08.284504 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] PATH_INFO: `/v3/users/da1310515be84efaa2fc3b0ee19f5044` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:08.286274 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1310515be84efaa2fc3b0ee19f5044)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:08.289047 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:08.342145 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1310515be84efaa2fc3b0ee19f5044'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.349293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1310515be84efaa2fc3b0ee19f5044'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.359259 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1310515be84efaa2fc3b0ee19f5044'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.359920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1310515be84efaa2fc3b0ee19f5044'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.360624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1310515be84efaa2fc3b0ee19f5044'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.379948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1386/2785] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:47:08 2022] DELETE /identity/v3/users/da1310515be84efaa2fc3b0ee19f5044 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:08.384657 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:08.385222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:08.385222 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:08.404935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:08.406172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:08.481838 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1400/2786] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:08.492525 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:08.495019 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:08.554319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56c70ef-c3d0-493e-b4d3-de79c68a7448 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:08.558455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:08.559040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:08.559348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:08.559612 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:08.569290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:08.571172 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:08.580966 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1387/2787] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:08.860875 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:08.862363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:08.874316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a7043c-db3e-42fc-ba8f-99a7847b64a3 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:08.879290 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:08.879932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:08.880206 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:08.880476 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] PATH_INFO: `/v3/projects/80a33c7355ec451c8ac20f4b187380b6` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:08.881602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a33c7355ec451c8ac20f4b187380b6)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:08.883549 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:08.931751 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a33c7355ec451c8ac20f4b187380b6', 'request_id': 'req-a9e29b8e-197d-4248-b483-88ae2e457c16'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.937278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a33c7355ec451c8ac20f4b187380b6', 'request_id': 'req-a9e29b8e-197d-4248-b483-88ae2e457c16'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.946129 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a33c7355ec451c8ac20f4b187380b6', 'request_id': 'req-a9e29b8e-197d-4248-b483-88ae2e457c16'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.957757 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] The token cache is being invalidate because project 80a33c7355ec451c8ac20f4b187380b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:47:08.958045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.958719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:08.960277 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1401/2788] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:47:08 2022] DELETE /identity/v3/projects/80a33c7355ec451c8ac20f4b187380b6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:08.964508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:08.974895 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ef263-53b3-4b36-aa77-523a7eb33e2b service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:09.000905 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:09.037157 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:09.037794 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:09.038072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:09.038319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] PATH_INFO: `/v3/users/dc10d5bf37804de2aa47887cc54bb965` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:09.039638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc10d5bf37804de2aa47887cc54bb965)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:09.041580 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:09.094056 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc10d5bf37804de2aa47887cc54bb965'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.102000 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc10d5bf37804de2aa47887cc54bb965'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.111321 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc10d5bf37804de2aa47887cc54bb965'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.111935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc10d5bf37804de2aa47887cc54bb965'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.112535 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc10d5bf37804de2aa47887cc54bb965'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.129515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1388/2789] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:47:08 2022] DELETE /identity/v3/users/dc10d5bf37804de2aa47887cc54bb965 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:09.435275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:09.437341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:09.459554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29b8e-197d-4248-b483-88ae2e457c16 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:09.524836 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:09.525441 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:09.525716 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:09.525984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] PATH_INFO: `/v3/projects/a438d405c5d241988dfdba2c22c55116` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:09.527140 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a438d405c5d241988dfdba2c22c55116)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:09.529167 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:09.554985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:09.585665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a438d405c5d241988dfdba2c22c55116', 'request_id': 'req-55a9d030-1254-4dc4-9e80-2da25f2c1d41'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.594134 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a438d405c5d241988dfdba2c22c55116', 'request_id': 'req-55a9d030-1254-4dc4-9e80-2da25f2c1d41'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.603235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a438d405c5d241988dfdba2c22c55116', 'request_id': 'req-55a9d030-1254-4dc4-9e80-2da25f2c1d41'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.616077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] The token cache is being invalidate because project a438d405c5d241988dfdba2c22c55116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:47:09.616366 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.617081 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.618681 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1402/2790] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:47:09 2022] DELETE /identity/v3/projects/a438d405c5d241988dfdba2c22c55116 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:09.622913 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:09.624466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:09.637671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870eed5-ba69-4ebf-aafa-cf279e164cc5 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:09.697564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:09.698201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:09.698474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:09.698739 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] PATH_INFO: `/v3/users/2e746bc7966c403c9d75bc34021841c9` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:09.699998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e746bc7966c403c9d75bc34021841c9)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:09.701944 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:09.755678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e746bc7966c403c9d75bc34021841c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.757068 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:09.765542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e746bc7966c403c9d75bc34021841c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.775202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e746bc7966c403c9d75bc34021841c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.776102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e746bc7966c403c9d75bc34021841c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.776729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-2d56fe82-92db-4143-b010-3e8ba241933b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e746bc7966c403c9d75bc34021841c9'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:09.796315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1389/2791] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:47:09 2022] DELETE /identity/v3/users/2e746bc7966c403c9d75bc34021841c9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:10.085815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.087436 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:10.101154 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a9d030-1254-4dc4-9e80-2da25f2c1d41 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.160450 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.161097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.161349 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.161652 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] PATH_INFO: `/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.162879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75e083c6bdb54f3d9e2445ccb4dc1b25)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.164994 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.214390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75e083c6bdb54f3d9e2445ccb4dc1b25', 'request_id': 'req-3fd1e734-c1f8-4408-b831-19bffb775ca4'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:10.220077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75e083c6bdb54f3d9e2445ccb4dc1b25', 'request_id': 'req-3fd1e734-c1f8-4408-b831-19bffb775ca4'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:10.228734 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75e083c6bdb54f3d9e2445ccb4dc1b25', 'request_id': 'req-3fd1e734-c1f8-4408-b831-19bffb775ca4'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:10.240019 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] The token cache is being invalidate because project 75e083c6bdb54f3d9e2445ccb4dc1b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:47:10.240301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:10.240955 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-3fd1e734-c1f8-4408-b831-19bffb775ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:10.242527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1403/2792] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:47:10 2022] DELETE /identity/v3/projects/75e083c6bdb54f3d9e2445ccb4dc1b25 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:10.247790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.248077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.248346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.270020 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:10.271206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:10.334634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1390/2793] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:10.344030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.344332 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.344587 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.365943 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:10.367198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:10.376008 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1404/2794] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:10.380216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.390206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae16061-bedf-4626-bcf8-d5b8d7ff9faa None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.395180 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.395781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.396066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.396335 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.396987 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:47:10.397293 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.398933 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.411483 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1391/2795] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:47:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:47:10.416040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.425905 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83373e45-2c23-4eb3-bf8b-cc72b5c1ab6e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.431979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.432583 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.432848 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.433128 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.433759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.435705 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.466880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1405/2796] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:47:10 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:10.471763 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.483446 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7f875-b629-4ebd-8d26-bb7f943701e6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.546617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.547419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.547689 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.547972 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.548604 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.550571 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.578699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1392/2797] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:47:10 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:10.583014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.594082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd836aa3-2481-4152-88b9-9a767eaf2c77 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.599384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.599997 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.600274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.600542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.601193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.602798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.616462 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1406/2798] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:47:10.621023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.632132 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f4995-1161-4d1b-a925-6cc9dd8c39c7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.638276 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.638907 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.639311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.639584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.640243 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.641867 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754850bb-d8a2-445a-b7b8-5463ec51ba2d admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.653440 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1393/2799] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:47:10.659679 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.667615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:10.674789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c749ac51-f425-4997-b878-c3669a1ab142 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.681621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.682294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.682562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.682831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] PATH_INFO: `/v3/projects/2be003b904c84e3bb6659dad80e0397c/users/edbe0e46c76c4d84a8af30d1fb124753/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.699384 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2be003b904c84e3bb6659dad80e0397c, user_id=edbe0e46c76c4d84a8af30d1fb124753, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.702058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.719856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1407/2800] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:47:10 2022] PUT /identity/v3/projects/2be003b904c84e3bb6659dad80e0397c/users/edbe0e46c76c4d84a8af30d1fb124753/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:10.730820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.731099 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.731390 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.761042 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] MFA Rules not processed for user `edbe0e46c76c4d84a8af30d1fb124753`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:10.762448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:10.782404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:10.829984 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1394/2801] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:10.843388 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:10.845892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:10.904954 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5573490b-8014-49e0-9fa4-a673bf5d2719 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:10.909315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:10.909904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:10.910180 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:10.910449 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:10.919181 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:10.920616 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4f905-0c47-4090-a45e-5c68645c5650 service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:10.929709 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1408/2802] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:20.826850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=18 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:20.925627 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:27.598101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:27.599839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:27.674383 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b5b48-e620-46c3-bba8-f1d8c9719a14 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:27.680305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:27.680939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:27.681212 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:27.681475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] PATH_INFO: `/v3/users/9650e14e8dfb4ed3a16498fdfb443a49` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:27.682736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9650e14e8dfb4ed3a16498fdfb443a49)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:27.684710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:27.741223 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9650e14e8dfb4ed3a16498fdfb443a49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:27.748351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9650e14e8dfb4ed3a16498fdfb443a49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:27.757262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9650e14e8dfb4ed3a16498fdfb443a49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:27.757873 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9650e14e8dfb4ed3a16498fdfb443a49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:27.758493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9650e14e8dfb4ed3a16498fdfb443a49'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:27.775419 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1395/2803] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:47:27 2022] DELETE /identity/v3/users/9650e14e8dfb4ed3a16498fdfb443a49 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:27.780417 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201e13-b063-456c-9a87-fd98e859519b None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:27.780938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201e13-b063-456c-9a87-fd98e859519b None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:27.780938 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be201e13-b063-456c-9a87-fd98e859519b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:27.801003 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-be201e13-b063-456c-9a87-fd98e859519b None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:27.802209 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-be201e13-b063-456c-9a87-fd98e859519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:27.867811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1409/2804] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:27.879460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:27.880957 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:27.949722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72f488-1b36-4b1f-9340-bb399aedf8e3 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:27.955420 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:27.956076 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:27.956348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:27.957272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:27.969124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:27.971008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f81a0ad-a12c-4366-a86a-f441ef6b985d service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:27.982512 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1396/2805] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:28.344986 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201e13-b063-456c-9a87-fd98e859519b None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.347005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-be201e13-b063-456c-9a87-fd98e859519b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:28.361774 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be201e13-b063-456c-9a87-fd98e859519b None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.368295 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.369404 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.375344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.375344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] PATH_INFO: `/v3/projects/f4d965940a72402b8706a0967ea1a243` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.375344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4d965940a72402b8706a0967ea1a243)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.375344 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.435232 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4d965940a72402b8706a0967ea1a243', 'request_id': 'req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:28.442856 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4d965940a72402b8706a0967ea1a243', 'request_id': 'req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:28.451949 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4d965940a72402b8706a0967ea1a243', 'request_id': 'req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:28.463602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] The token cache is being invalidate because project f4d965940a72402b8706a0967ea1a243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:47:28.463893 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:28.464572 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-cbc60f1d-d5bb-4d1b-bdb5-a66b7cc52241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:28.466408 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1410/2806] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:47:28 2022] DELETE /identity/v3/projects/f4d965940a72402b8706a0967ea1a243 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:28.472344 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.472732 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.473202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.495181 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:28.496515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:28.514154 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:28.558189 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1397/2807] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:28.567113 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.567390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.567722 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.588597 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:28.589814 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:28.599037 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1411/2808] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:28.603445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.614002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537c1b7-c66a-46d7-8750-3822e1eeb566 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.619893 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.620521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.620800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.621051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.621744 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:47:28.622023 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.623668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.640079 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1398/2809] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:47:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:47:28.641426 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.656047 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bfeea-fc7b-4e1a-a6f0-f54fd76d4c70 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.664544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.665360 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.665708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.666049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.666764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.671866 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.700976 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1412/2810] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:47:28 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:28.705262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.718699 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea44f-f875-4a15-90d1-5a5f33f09bf6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.778647 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.779310 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.779591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.779844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.780488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.782429 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.805184 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1399/2811] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:47:28 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:47:28.810818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.821691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd00466-e55f-41ae-afa8-bfbea7a25da3 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.827288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.827990 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.828258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.828522 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.829157 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.830773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.841869 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1413/2812] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:47:28.846434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.858231 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95de5c-3c05-4a1b-8c95-cd5c90e69991 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.864733 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.865351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.865630 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.865900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.866550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.868209 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48752c3-9986-44c1-8ddb-9611f9ad0897 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.879928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1400/2813] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:47:28.884397 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:28.895822 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a70756-4ecb-4f58-beac-aed2f3025e43 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:28.903520 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:28.904471 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.904829 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.905186 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] PATH_INFO: `/v3/projects/5a479cfd62f048a5aae793d449e9e9d4/users/9cf127f9aa17428bb43df644e17434c6/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.922925 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a479cfd62f048a5aae793d449e9e9d4, user_id=9cf127f9aa17428bb43df644e17434c6, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:28.925625 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:28.941252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1414/2814] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:47:28 2022] PUT /identity/v3/projects/5a479cfd62f048a5aae793d449e9e9d4/users/9cf127f9aa17428bb43df644e17434c6/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:28.946503 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c02b4-2133-4dac-a2d7-8738cb632419 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:28.946782 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c02b4-2133-4dac-a2d7-8738cb632419 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:28.947051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c02b4-2133-4dac-a2d7-8738cb632419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:28.967550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-796c02b4-2133-4dac-a2d7-8738cb632419 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:28.968759 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-796c02b4-2133-4dac-a2d7-8738cb632419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:29.040605 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1401/2815] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:29.050809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:29.052926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:29.084124 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:29.122773 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0f8c0e-fdf8-410e-bb0c-29be793fe7c9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:29.126988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:29.127615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:29.127892 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:29.128159 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:29.137552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:29.138980 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:29.148406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1415/2816] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:29.820051 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315073-dfd9-47c9-9ed6-429a0e02b737 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:29.820570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315073-dfd9-47c9-9ed6-429a0e02b737 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:29.820570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d315073-dfd9-47c9-9ed6-429a0e02b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:29.850409 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-6d315073-dfd9-47c9-9ed6-429a0e02b737 None None] MFA Rules not processed for user `9cf127f9aa17428bb43df644e17434c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:29.851551 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6d315073-dfd9-47c9-9ed6-429a0e02b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:29.869455 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:29.920454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1402/2817] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:29.932694 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:29.934143 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:29.993168 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb99a42-65f6-4892-a15b-0f5b33687d2a service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:29.996926 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:29.997538 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:29.997811 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:29.998076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:30.006907 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:30.008375 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:30.017275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1416/2818] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:31.437569 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:31.438554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:31.438811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:31.459437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] MFA Rules not processed for user `9cf127f9aa17428bb43df644e17434c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:31.460634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:31.468948 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1403/2819] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:33.063770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:33.065356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:33.142612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2767f0-4f1f-4829-8321-f5212353803f service cinder] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:33.147677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:33.148279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:33.148540 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:33.148818 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:33.150746 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:33.162799 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:33.164480 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:33.174762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1417/2820] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:41.454119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=22 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:43.170666 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=75 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:45.522840 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:45.525245 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:45.617160 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e2254b-89af-46a0-a47d-06530e2ae6d3 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:45.623573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:45.624224 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:45.624501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:45.624771 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:45.626531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:45.639036 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:45.640653 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd26e081-a5b0-47fb-be27-d96449099f9d service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:45.651053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1404/2821] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:55.646822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=34 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:47:56.003958 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:56.005574 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:56.020473 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49041de-135b-44bd-942f-56e3fe7a4548 service placement] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:56.025914 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:56.026544 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:56.026823 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:56.027103 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] PATH_INFO: `/v3/users/edbe0e46c76c4d84a8af30d1fb124753` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:56.028323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbe0e46c76c4d84a8af30d1fb124753)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:56.030252 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:56.083888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbe0e46c76c4d84a8af30d1fb124753'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.091696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbe0e46c76c4d84a8af30d1fb124753'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.092455 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbe0e46c76c4d84a8af30d1fb124753'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.093075 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbe0e46c76c4d84a8af30d1fb124753'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.101761 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbe0e46c76c4d84a8af30d1fb124753'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.121155 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1418/2822] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:47:56 2022] DELETE /identity/v3/users/edbe0e46c76c4d84a8af30d1fb124753 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:47:56.126059 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:56.126348 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:56.126624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:56.148029 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:47:56.149306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:56.218196 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1405/2823] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:47:56.229456 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:56.231881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:56.295068 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bfffeb-f2aa-4813-89f7-289d33eb2777 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:56.299472 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:56.300079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:56.300353 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:56.300618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:56.312567 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:56.314077 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:56.326803 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1419/2824] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:47:56.626727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:47:56.628267 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:47:56.641236 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcda21cc-5b54-4043-b8b7-9f4212d2a4b5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:47:56.646902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:47:56.647573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:47:56.647849 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:47:56.648117 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] PATH_INFO: `/v3/projects/2be003b904c84e3bb6659dad80e0397c` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:47:56.653174 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2be003b904c84e3bb6659dad80e0397c)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:47:56.655173 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:47:56.707327 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2be003b904c84e3bb6659dad80e0397c', 'request_id': 'req-27c00624-c128-45ea-ab13-6061af394475'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.712619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2be003b904c84e3bb6659dad80e0397c', 'request_id': 'req-27c00624-c128-45ea-ab13-6061af394475'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.723449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2be003b904c84e3bb6659dad80e0397c', 'request_id': 'req-27c00624-c128-45ea-ab13-6061af394475'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.736701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] The token cache is being invalidate because project 2be003b904c84e3bb6659dad80e0397c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:47:56.736986 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.737671 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-27c00624-c128-45ea-ab13-6061af394475 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:47:56.739246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1406/2825] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:47:56 2022] DELETE /identity/v3/projects/2be003b904c84e3bb6659dad80e0397c => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:48:06.320564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:48:06.732481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:48:44.944182 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:48:44.945692 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:48:45.025005 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fda64-d392-48c7-b97c-ff3d12d338f4 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:48:45.087364 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:48:45.087979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:48:45.088258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:48:45.088531 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:48:45.090326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:48:45.103281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:48:45.104872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:48:45.115791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1420/2826] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:48:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:48:55.110966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=58 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:48:56.663806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:48:56.664394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:48:56.664394 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:48:56.684816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] MFA Rules not processed for user `9cf127f9aa17428bb43df644e17434c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:48:56.686035 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:48:56.743363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1407/2827] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:48:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:48:57.003198 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:48:57.004987 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:48:57.074275 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9390c0e-52b2-4141-a99e-f879554c666c service placement] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:48:57.077884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:48:57.078477 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:48:57.078752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:48:57.079015 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:48:57.080767 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:48:57.091052 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:48:57.092524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd81a57-a9b8-490d-a029-465442c38a46 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:48:57.101512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1421/2828] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:48:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:49:06.736210 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=25 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:07.097557 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=26 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:29.042278 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:29.043843 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:29.128305 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdcb07-7d33-4b08-8f39-8c176c2ac2c5 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:29.136990 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:29.137703 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:29.138043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:29.139531 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:29.141743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:29.157395 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:29.159119 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:29.172428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1408/2829] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:49:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:49:38.653483 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:38.654278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:38.654278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:38.683909 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] MFA Rules not processed for user `6afd5333a03e4e098460fbbeef927d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:38.685162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:38.759231 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1422/2830] 10.222.0.33 () {58 vars in 967 bytes} [Mon Aug 8 13:49:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:49:38.807498 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:38.809275 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:38.889765 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218436ef-c243-429e-a235-2af7a5181da9 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:38.895220 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec66fef9dc274223b34659e369063ced', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:38.895869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:38.896139 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:38.896433 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:38.905696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:38.907342 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:38.916822 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1409/2831] 10.222.0.33 () {60 vars in 1384 bytes} [Mon Aug 8 13:49:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 08 13:49:42.765419 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:42.766904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:42.783877 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1d82e2-d0c8-4839-b2a6-89369c573cc9 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:42.789323 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:42.790667 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:42.791009 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:42.791415 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:42.793221 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:42.805778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:42.807424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:42.818675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1423/2832] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:49:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:49:46.351295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:46.352962 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:46.435928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffdf7db-ca7e-496b-a283-a4657bd11921 service swift] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:46.443227 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:46.443902 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:46.444183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:46.444463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] PATH_INFO: `/v3/users/14aab8af2e6b4201b0951199de8156e8` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:46.445705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=14aab8af2e6b4201b0951199de8156e8)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:46.447820 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:46.469358 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:46.511779 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14aab8af2e6b4201b0951199de8156e8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.522374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14aab8af2e6b4201b0951199de8156e8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.533537 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14aab8af2e6b4201b0951199de8156e8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.534461 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14aab8af2e6b4201b0951199de8156e8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.535206 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14aab8af2e6b4201b0951199de8156e8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.556922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1410/2833] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:49:46 2022] DELETE /identity/v3/users/14aab8af2e6b4201b0951199de8156e8 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:46.753266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:46.765489 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db341d23-cb21-45f9-ab0e-062cad91e7b8 service placement] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:46.829620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:46.830326 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:46.830646 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:46.830962 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] PATH_INFO: `/v3/projects/554bfc93b1a44d57a5e8e5868cd2eed6` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:46.832511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=554bfc93b1a44d57a5e8e5868cd2eed6)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:46.834586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:46.887265 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '554bfc93b1a44d57a5e8e5868cd2eed6', 'request_id': 'req-6436cf81-01fe-4915-8761-0d037aa3e2a5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.893612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '554bfc93b1a44d57a5e8e5868cd2eed6', 'request_id': 'req-6436cf81-01fe-4915-8761-0d037aa3e2a5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.902655 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '554bfc93b1a44d57a5e8e5868cd2eed6', 'request_id': 'req-6436cf81-01fe-4915-8761-0d037aa3e2a5'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.915061 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] The token cache is being invalidate because project 554bfc93b1a44d57a5e8e5868cd2eed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:49:46.915383 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.916062 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:46.917618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1424/2834] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:49:46 2022] DELETE /identity/v3/projects/554bfc93b1a44d57a5e8e5868cd2eed6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:46.922114 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:46.924167 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:46.944485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf764cc-b1bb-4125-9184-e224a74d2fc6 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.008998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.009698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.009968 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.010290 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] PATH_INFO: `/v3/users/ce1aef1ef76f4d75a44eb7f3d74a70d8` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.011642 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1aef1ef76f4d75a44eb7f3d74a70d8)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.013808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.065033 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1aef1ef76f4d75a44eb7f3d74a70d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.073920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1aef1ef76f4d75a44eb7f3d74a70d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.084616 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1aef1ef76f4d75a44eb7f3d74a70d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.085602 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1aef1ef76f4d75a44eb7f3d74a70d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.086219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-dee458ae-c0c3-4ae5-bdc8-3fde7fa35eff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1aef1ef76f4d75a44eb7f3d74a70d8'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.107697 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1411/2835] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:49:46 2022] DELETE /identity/v3/users/ce1aef1ef76f4d75a44eb7f3d74a70d8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:47.298582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.300097 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:47.318058 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6436cf81-01fe-4915-8761-0d037aa3e2a5 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.339881 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:47.381812 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.382620 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.382902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.383279 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] PATH_INFO: `/v3/projects/c9163203450845d19ba4ccac40125db4` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.384569 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9163203450845d19ba4ccac40125db4)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.386661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.442511 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9163203450845d19ba4ccac40125db4', 'request_id': 'req-f723d0b0-d34c-49af-a2c0-a9ba15056847'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.448915 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9163203450845d19ba4ccac40125db4', 'request_id': 'req-f723d0b0-d34c-49af-a2c0-a9ba15056847'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.458067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9163203450845d19ba4ccac40125db4', 'request_id': 'req-f723d0b0-d34c-49af-a2c0-a9ba15056847'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.470804 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] The token cache is being invalidate because project c9163203450845d19ba4ccac40125db4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:49:47.471402 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.472141 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-f723d0b0-d34c-49af-a2c0-a9ba15056847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:49:47.473783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1425/2836] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:49:47 2022] DELETE /identity/v3/projects/c9163203450845d19ba4ccac40125db4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:47.479673 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.480214 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.480315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.500792 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:47.502087 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:47.511489 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:47.573084 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1412/2837] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:49:47.584175 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.584453 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.585211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.613936 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:47.615760 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:47.625661 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1426/2838] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:49:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:49:47.630756 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.641171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93034a15-771f-463e-bac2-4fd6180a7cd7 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.646696 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.647371 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.647649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.647919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] PATH_INFO: `/v3/domains` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.648609 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80187) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:49:47.648906 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.650648 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.666043 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1413/2839] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 08 13:49:47.672211 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.689120 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f2faf-b456-430d-868a-a81d834e6fd0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.696988 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.698016 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.698542 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.699007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.700214 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.703977 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.749798 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1427/2840] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:49:47 2022] POST /identity/v3/projects => generated 379 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:49:47.755011 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.766053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fca2b-1831-4271-922f-df0f8520d8db admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.831445 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.832222 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.832523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.832826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.833523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.835872 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.863340 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1414/2841] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:49:47 2022] POST /identity/v3/users => generated 290 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:49:47.867512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.879945 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0021b9-a8fe-48c3-b457-ebdedeca82d0 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.885801 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.886424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.886730 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.887025 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.887724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.889389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.900791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1428/2842] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:49:47.905118 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.916965 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce5a31-5f81-467e-9a2f-63b5a6185817 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.922142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.922760 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.923038 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.923341 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.923999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.925622 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58385cda-28a0-4a35-859f-5b9c17f1769b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:47.937283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1415/2843] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:49:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:49:47.941719 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:47.958331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d4f6f2-173d-4a7a-8068-b15fcaf41c59 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:47.963745 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:47.964339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:47.964615 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:47.964891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] PATH_INFO: `/v3/projects/a019b128831a47b7996b3d3e9d6e5652/users/d4bb35df2ffd48bf965425afcdbc42fd/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:47.990631 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a019b128831a47b7996b3d3e9d6e5652, user_id=d4bb35df2ffd48bf965425afcdbc42fd, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:47.993357 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:48.012072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1429/2844] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:49:47 2022] PUT /identity/v3/projects/a019b128831a47b7996b3d3e9d6e5652/users/d4bb35df2ffd48bf965425afcdbc42fd/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:48.017345 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7f88-5e15-4554-860d-4b74e3cf4605 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:48.017643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7f88-5e15-4554-860d-4b74e3cf4605 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:48.017918 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4c7f88-5e15-4554-860d-4b74e3cf4605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:48.039985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-de4c7f88-5e15-4554-860d-4b74e3cf4605 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:48.041588 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-de4c7f88-5e15-4554-860d-4b74e3cf4605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:48.110844 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1416/2845] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:49:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:49:48.123000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:48.125516 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:48.186100 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:48.191510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b995b-21ae-433f-b107-72ac25cdead4 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:48.197953 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:48.198871 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:48.199219 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:48.199764 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:48.210119 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:48.211645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:48.223136 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1430/2846] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:49:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:49:53.147828 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.152511 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.153101 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.190453 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] MFA Rules not processed for user `d4bb35df2ffd48bf965425afcdbc42fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:53.191806 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:53.201839 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:53.276929 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1417/2847] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:49:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:49:53.278647 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.290602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864926-1c89-4c77-a716-4e230897cd57 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.295791 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.296524 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.296964 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.297308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.298040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.300226 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.332752 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1431/2848] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:49:53 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:49:53.337195 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.349608 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc86f0c-3a00-4a87-8ee9-0ebc8fff7a0c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.424858 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.425661 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.425939 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.426288 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] PATH_INFO: `/v3/users` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.426941 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.429463 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.471686 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1418/2849] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:49:53 2022] POST /identity/v3/users => generated 295 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:49:53.473235 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.485130 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66abbd-1445-421e-ae1c-0e102148c6eb admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.491262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.491922 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.492189 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.492458 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.493125 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.495014 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.509142 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1432/2850] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:49:53.514303 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.525781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c709348-f33b-4996-8586-1c40467b1a54 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.530816 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.531577 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.531865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.532153 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] PATH_INFO: `/v3/projects/11c8cd6d1c594e89b417c04221867541/users/d3bf4100e8684c099d78e5e0eed3b4c5/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.549436 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c8cd6d1c594e89b417c04221867541, user_id=d3bf4100e8684c099d78e5e0eed3b4c5, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.552430 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.568557 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1419/2851] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:49:53 2022] PUT /identity/v3/projects/11c8cd6d1c594e89b417c04221867541/users/d3bf4100e8684c099d78e5e0eed3b4c5/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:53.572727 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.585269 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167cb41-6f26-4389-aa27-35010a70a96f admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.658969 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.659592 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.659891 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.660207 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.660902 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.662778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8433bac0-d45f-420c-a10c-c3977748592d admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.678492 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1433/2852] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:49:53 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:49:53.680767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:53.695757 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cb5ea7-70b9-4b54-927f-86be95eacfeb admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:53.702969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:53.703711 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:53.704002 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:53.704248 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] PATH_INFO: `/v3/projects/11c8cd6d1c594e89b417c04221867541/users/d3bf4100e8684c099d78e5e0eed3b4c5/roles/193a5d91676042449492edcffd7d1760` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:53.706331 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11c8cd6d1c594e89b417c04221867541, user_id=d3bf4100e8684c099d78e5e0eed3b4c5, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:53.710492 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:53.726485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1420/2853] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:49:53 2022] PUT /identity/v3/projects/11c8cd6d1c594e89b417c04221867541/users/d3bf4100e8684c099d78e5e0eed3b4c5/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:49:59.070281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:59.070941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:59.070941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:59.111418 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] MFA Rules not processed for user `d3bf4100e8684c099d78e5e0eed3b4c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:49:59.113072 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:59.195762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1434/2854] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:49:59 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:49:59.214518 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:49:59.216940 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:49:59.302869 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:49:59.339106 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58486a-f903-488f-ae63-1a3060d21332 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:49:59.353427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:49:59.354208 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:49:59.356316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:49:59.356655 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:49:59.441164 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:49:59.442784 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:49:59.452523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1421/2855] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:49:59 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:08.169334 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:08.170900 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:08.224087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:08.247292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4be25-1e5e-4d6a-82a9-1ca2f27be8cd None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:08.252274 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:08.252923 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.253197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.253464 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] PATH_INFO: `/v3/users/9cf127f9aa17428bb43df644e17434c6` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.254675 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cf127f9aa17428bb43df644e17434c6)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:08.256685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:08.313190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cf127f9aa17428bb43df644e17434c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.320534 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cf127f9aa17428bb43df644e17434c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.321249 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cf127f9aa17428bb43df644e17434c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.321963 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cf127f9aa17428bb43df644e17434c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.334348 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-c4c873b8-c2ca-4b86-94e8-b40fddc1d5e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cf127f9aa17428bb43df644e17434c6'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.368339 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1435/2856] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:50:08 2022] DELETE /identity/v3/users/9cf127f9aa17428bb43df644e17434c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:50:08.555698 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:08.567241 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55839f-54c9-48e9-aeac-c10ffe745b56 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:08.629272 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:08.629812 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.630092 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.630362 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] PATH_INFO: `/v3/projects/5a479cfd62f048a5aae793d449e9e9d4` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.631493 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a479cfd62f048a5aae793d449e9e9d4)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:08.633472 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:08.687743 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a479cfd62f048a5aae793d449e9e9d4', 'request_id': 'req-93d13847-a9c5-476c-aab5-2a2777a0bbec'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.696527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a479cfd62f048a5aae793d449e9e9d4', 'request_id': 'req-93d13847-a9c5-476c-aab5-2a2777a0bbec'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.706401 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a479cfd62f048a5aae793d449e9e9d4', 'request_id': 'req-93d13847-a9c5-476c-aab5-2a2777a0bbec'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.719510 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] The token cache is being invalidate because project 5a479cfd62f048a5aae793d449e9e9d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:50:08.719778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.720550 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-93d13847-a9c5-476c-aab5-2a2777a0bbec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:50:08.722479 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1422/2857] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:50:08 2022] DELETE /identity/v3/projects/5a479cfd62f048a5aae793d449e9e9d4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:50:08.728495 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.729004 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.729303 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.754771 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:08.756174 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:08.818691 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1436/2858] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:50:08.827448 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.827646 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.827920 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.849403 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:08.850710 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:08.864554 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1423/2859] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:50:08.869076 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:08.879552 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b85409d-bf91-4249-a6ab-694d6587e0ae None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:08.884788 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:08.885399 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.885674 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.885941 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.886591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80188) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 08 13:50:08.886886 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:08.888527 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:08.900759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1437/2860] 10.222.0.33 () {58 vars in 1153 bytes} [Mon Aug 8 13:50:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 08 13:50:08.905242 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:08.914900 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1013b83c-a22b-43ed-9722-97154d64bb76 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:08.921536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:08.922237 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:08.922578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:08.922922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:08.923731 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:08.925866 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:08.955030 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:08.962488 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1424/2861] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:50:08 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:50:08.968577 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:08.979882 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b61f77-07d3-4a8d-a9be-09d1e8673377 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:09.029311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:09.045460 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:09.046057 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.046331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.046602 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.047258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:09.049199 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:09.076695 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1438/2862] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:50:08 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:50:09.078285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:09.090100 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ddf6a-a871-4f8d-ac8d-57daba27d81e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:09.095755 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:09.096721 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.096995 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.097262 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.097908 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:09.099572 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:09.110460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1425/2863] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:50:09.114724 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:09.126491 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcb9745-1639-430d-b05d-5f98d3eb03e9 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:09.132391 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:09.132928 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.133230 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.133502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.134172 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:09.135842 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e515ea-74d9-47bf-8d54-c9cf5890bb7a admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:09.148380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1439/2864] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:50:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 08 13:50:09.155399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:09.168997 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5a2161-7a40-4a75-8a36-3e43978fdafc admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:09.176086 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:09.176850 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.177162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.177437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] PATH_INFO: `/v3/projects/e6c67730fcf64392b047dd178ee72db8/users/74a63ce39dd942f9abb785d05cbd8c2b/roles/2cb0880dbcea4e93aefe6370c68adecf` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.194168 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c67730fcf64392b047dd178ee72db8, user_id=74a63ce39dd942f9abb785d05cbd8c2b, role_id=2cb0880dbcea4e93aefe6370c68adecf)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:09.196985 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:09.212315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1426/2865] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:50:09 2022] PUT /identity/v3/projects/e6c67730fcf64392b047dd178ee72db8/users/74a63ce39dd942f9abb785d05cbd8c2b/roles/2cb0880dbcea4e93aefe6370c68adecf => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:50:09.217634 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bac3d-ea99-4745-9827-b5b63962ef55 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.218127 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bac3d-ea99-4745-9827-b5b63962ef55 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.218554 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bac3d-ea99-4745-9827-b5b63962ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.240806 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-cd7bac3d-ea99-4745-9827-b5b63962ef55 None None] MFA Rules not processed for user `69e577935e2a4950900c965d118c5e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:09.242191 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-cd7bac3d-ea99-4745-9827-b5b63962ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:09.308031 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1440/2866] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 08 13:50:09.318761 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:09.320942 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:09.387523 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a4db9-757b-49f5-8d39-021de53dbf0a admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:09.391818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:09.392434 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:09.392705 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:09.392971 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:09.414702 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:09.416631 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:09.431116 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1427/2867] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:13.589533 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:13.590110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:13.590110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:13.620837 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] MFA Rules not processed for user `74a63ce39dd942f9abb785d05cbd8c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:13.622030 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:13.693087 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1441/2868] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:50:13.697678 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:13.707524 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe7555-af1e-493b-a3a0-3ba6ca0aa122 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:13.712796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:13.713416 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:13.713694 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:13.713969 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:13.714613 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:13.716585 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:13.755473 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1428/2869] 10.222.0.33 () {60 vars in 1153 bytes} [Mon Aug 8 13:50:13 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 08 13:50:13.759586 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:13.771622 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d3659-da4d-44c9-a880-09fc18142ef0 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:13.829400 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:13.838083 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:13.838703 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:13.838984 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:13.839281 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] PATH_INFO: `/v3/users` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:13.839951 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:13.841947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:13.866831 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1442/2870] 10.222.0.33 () {60 vars in 1144 bytes} [Mon Aug 8 13:50:13 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 08 13:50:13.870992 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:13.882265 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d657a8-d3a8-4c03-9ea1-1caa57bd8a04 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:13.887570 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:13.888202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:13.888516 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:13.888791 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:13.889536 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:13.891284 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:13.903143 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1429/2871] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:50:13.907559 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:13.918305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dca518-2728-493a-836f-062cb4abfede admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:13.923070 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:13.923778 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:13.924060 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:13.924363 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] PATH_INFO: `/v3/projects/b9babb62146549a9aae03a243ecd3e67/users/336e9ea413e542aaa06487773f74bebd/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:13.940564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9babb62146549a9aae03a243ecd3e67, user_id=336e9ea413e542aaa06487773f74bebd, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:13.943447 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:13.962373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1443/2872] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:50:13 2022] PUT /identity/v3/projects/b9babb62146549a9aae03a243ecd3e67/users/336e9ea413e542aaa06487773f74bebd/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:50:13.963767 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:13.974396 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e31ac4-5c49-4d05-93b4-e5914ff9589e admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:14.043888 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:14.044508 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:14.044790 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:14.045053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] PATH_INFO: `/v3/roles` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:14.045720 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:14.047397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa28ba09-d5d3-492e-a3ae-6dd3bf17e356 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:14.057664 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1430/2873] 10.222.0.33 () {58 vars in 1122 bytes} [Mon Aug 8 13:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 08 13:50:14.062267 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:14.074341 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fa7a18-2fdc-4ecf-a738-0c1b00621df7 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:14.079718 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:14.080380 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] REQUEST_METHOD: `PUT` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:14.080660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:14.080932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] PATH_INFO: `/v3/projects/b9babb62146549a9aae03a243ecd3e67/users/336e9ea413e542aaa06487773f74bebd/roles/193a5d91676042449492edcffd7d1760` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:14.083193 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9babb62146549a9aae03a243ecd3e67, user_id=336e9ea413e542aaa06487773f74bebd, role_id=193a5d91676042449492edcffd7d1760)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:14.086215 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:14.096957 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1444/2874] 10.222.0.33 () {60 vars in 1483 bytes} [Mon Aug 8 13:50:14 2022] PUT /identity/v3/projects/b9babb62146549a9aae03a243ecd3e67/users/336e9ea413e542aaa06487773f74bebd/roles/193a5d91676042449492edcffd7d1760 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:50:18.380045 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ef1e-6e5b-45a0-a72f-f1602e436391 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:18.380591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ef1e-6e5b-45a0-a72f-f1602e436391 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:18.385449 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ef1e-6e5b-45a0-a72f-f1602e436391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:18.430786 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-31b3ef1e-6e5b-45a0-a72f-f1602e436391 None None] MFA Rules not processed for user `336e9ea413e542aaa06487773f74bebd`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:18.432591 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-31b3ef1e-6e5b-45a0-a72f-f1602e436391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:18.465171 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:18.556644 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1431/2875] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:50:18.567908 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:18.570315 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:18.670573 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08ca91d-207b-4401-a42f-354d7d8a8884 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:18.676406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:18.677027 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:18.677300 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:18.677713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:18.745325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:18.747045 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f216cb-e08a-44fe-82a6-152078ac688b service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:18.757786 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1445/2876] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:19.804818 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:19.805296 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:19.805397 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:19.825981 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] MFA Rules not processed for user `74a63ce39dd942f9abb785d05cbd8c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:19.827306 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:19.834329 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1432/2877] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:50:23.481223 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:23.481698 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:23.481785 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:23.501110 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] MFA Rules not processed for user `d4bb35df2ffd48bf965425afcdbc42fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:50:23.502340 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:23.548638 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:23.569556 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1446/2878] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:50:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:50:23.612131 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:23.613584 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:23.675319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d374db8-bc3a-4843-8f4b-cf6090893cf2 None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:23.680219 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:23.680842 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:23.681149 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:23.681422 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:23.683399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:23.698374 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:23.699870 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:23.712048 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1433/2879] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:24.315213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:24.316904 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:24.385195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f915f824-3cca-4b8f-a9d4-b6ca736a0652 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:24.389035 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:24.389645 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:24.389916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:24.390197 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:24.391982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:24.402082 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:24.403676 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:24.411950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1447/2880] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:24.939573 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:24.950102 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcde1bfb-16d0-4fa4-9df9-b515fac3bfde service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:24.953932 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:24.954701 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:24.955040 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:24.955399 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:24.957295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:24.972998 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:24.974460 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:24.987095 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1434/2881] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:34.409054 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=31 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:34.981781 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=55 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:42.024259 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:42.033152 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2bae18-2d4c-4f98-a4a8-14e4f25ffcda service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:42.036824 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:42.037420 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:42.037696 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:42.037966 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:42.039699 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:42.111071 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:42.112623 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:42.122808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1448/2882] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:50:52.118629 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=30 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:50:58.925560 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:50:58.934617 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7554a497-0912-40c5-a9e7-781060c79053 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:50:58.938547 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:50:58.939177 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:50:58.939457 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:50:58.939709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:50:58.941564 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:50:58.951619 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:50:58.953146 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:50:58.961912 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1435/2883] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:50:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:51:01.629067 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:51:01.638770 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0eda9-9c68-46c5-bd57-316de489c556 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:51:01.643195 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:51:01.643789 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:51:01.644040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:51:01.644324 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:51:01.654919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:51:01.656389 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8009f4-6a3b-43a9-a23a-be61924e8596 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:51:01.669490 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1449/2884] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:51:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:51:08.958041 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=6 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:51:11.662288 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:53:01.029610 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:53:01.045808 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533200bf-d064-4158-93a1-26742ac184b0 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:53:01.056295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:53:01.057067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:01.057351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:01.057675 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:01.061475 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:01.075796 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:53:01.077443 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:53:01.092097 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1436/2885] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:53:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:53:02.990213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:02.990748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:02.990748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:03.012262 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] MFA Rules not processed for user `d4bb35df2ffd48bf965425afcdbc42fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:53:03.013469 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:03.021665 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1450/2886] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:53:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:53:03.812517 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:53:03.814069 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:03.889413 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c243665-cd23-48db-85eb-09daebc21e14 service nova] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:53:03.894538 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:53:03.895183 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:03.895465 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:03.895737 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:03.897452 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:03.908659 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:53:03.910235 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021c0ba-f0eb-4331-af56-fa42a8a588c0 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:53:03.920063 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1437/2887] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:53:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:53:13.006114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=3 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:53:13.916651 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=38 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:53:19.240431 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:53:19.242099 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:19.260316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197dba9-d3eb-442d-afe7-97e4d568c02e None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:53:19.265916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:53:19.266591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:19.266872 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:19.267178 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:19.269519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:19.283305 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:53:19.284919 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd796c-ac15-472d-ab09-e2a9e9ec8a2e service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:53:19.296017 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1451/2888] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:53:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:53:29.291448 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=6 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:53:32.096124 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650441-30fa-43ff-9115-984acd1e0e9c None None] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.096789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650441-30fa-43ff-9115-984acd1e0e9c None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.096789 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7650441-30fa-43ff-9115-984acd1e0e9c None None] PATH_INFO: `/` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.098007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1438/2889] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:53:32.103783 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171943e9-ee6c-479d-aad5-18994c3e41b1 None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.104282 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171943e9-ee6c-479d-aad5-18994c3e41b1 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.104487 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171943e9-ee6c-479d-aad5-18994c3e41b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.144759 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-171943e9-ee6c-479d-aad5-18994c3e41b1 None None] MFA Rules not processed for user `a36d4232444f46c08f950d19ae395643`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:53:32.146000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-171943e9-ee6c-479d-aad5-18994c3e41b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:32.153321 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1452/2890] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:53:32.165491 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650441-30fa-43ff-9115-984acd1e0e9c None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:53:32.180025 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7650441-30fa-43ff-9115-984acd1e0e9c None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:53:32.185999 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:53:32.186734 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.187013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.187311 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.195466 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:53:32.196923 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d4128-5826-4467-970c-22ca0295a92a service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:53:32.206980 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1439/2891] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:53:32.292609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd4aab-8322-47b8-b91b-a0c21dd5e143 None None] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.292894 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd4aab-8322-47b8-b91b-a0c21dd5e143 None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.293162 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd4aab-8322-47b8-b91b-a0c21dd5e143 None None] PATH_INFO: `/` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.294582 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1453/2892] 10.222.0.33 () {54 vars in 921 bytes} [Mon Aug 8 13:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 08 13:53:32.300240 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] REQUEST_METHOD: `POST` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.300515 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.300826 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.342162 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.auth.core [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] MFA Rules not processed for user `8e18ece85f014104b3c8fe92355b18a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80187) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:53:32.343437 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:53:32.354221 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1440/2893] 10.222.0.33 () {58 vars in 1018 bytes} [Mon Aug 8 13:53:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:53:32.365266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd4aab-8322-47b8-b91b-a0c21dd5e143 None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:53:32.377479 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd4aab-8322-47b8-b91b-a0c21dd5e143 None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:53:32.381839 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:53:32.382435 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:53:32.382708 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:53:32.382975 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:53:32.393864 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:53:32.395319 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:53:32.406610 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1454/2894] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:53:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:53:42.338776 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=15 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:53:42.401748 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=15 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:54:29.872715 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:54:29.936315 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad40c71-882e-4716-93bb-f2d07b07d81f None None] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:54:29.945077 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:54:29.945684 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:54:29.945954 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:54:29.946226 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:54:29.948217 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:54:29.982005 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:54:29.983640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:54:29.994477 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1441/2895] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:54:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:54:39.989769 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=36 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:12.895630 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:12.928292 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce60dc99-e573-4a4f-92db-90a670e470ab service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:12.932289 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:12.932888 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:12.933163 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:12.933433 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:12.935580 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:12.947325 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:12.948754 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538137b-ee1d-480f-9c7b-6d716df39bbf service cinder] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:12.958306 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1455/2896] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:18.917722 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:18.943316 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6123f070-2156-489a-bdd0-ae04675f9361 service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:18.947405 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:18.948008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:18.948280 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:18.948542 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:18.958007 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:18.959474 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:18.968851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1442/2897] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:22.953671 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=15 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:25.165648 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] REQUEST_METHOD: `POST` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:25.166331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:25.166331 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:25.193753 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.auth.core [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] MFA Rules not processed for user `d4bb35df2ffd48bf965425afcdbc42fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80188) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 08 13:55:25.195606 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:25.206808 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1456/2898] 10.222.0.33 () {58 vars in 958 bytes} [Mon Aug 8 13:55:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 08 13:55:25.246638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:25.261634 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57d5352-31c1-42b8-84d4-6dc0f4451985 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:25.267643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '656cb98cf9b147f993f2cd31ec4ea55c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:25.268251 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:25.268529 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:25.268800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:25.288624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:25.290066 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:25.315292 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1443/2899] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:25.663114 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:25.673880 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eac4eb-5bee-47c4-9760-3c0a79f5747f None None] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:25.677619 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:25.678451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:25.678890 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:25.679406 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:25.685370 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:25.695301 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8508ec3-7458-4123-8696-c5e76550397a service cinder] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:25.699246 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:25.699936 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:25.700201 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:25.700670 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:25.709975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:25.711501 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:25.721521 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1444/2900] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 08 13:55:25.765618 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:25.767308 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:25.788932 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1457/2901] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:26.207425 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:26.216363 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1087f117-f048-45c6-8ba4-0ff5ad97b1ae service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:26.220772 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:26.221595 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:26.221938 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:26.222351 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:26.235291 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:26.236879 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:26.265178 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1445/2902] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:29.321985 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:29.334266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a39daa-88c1-43ee-be74-b5ebaed140b8 service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:29.340809 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:29.341437 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:29.341715 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:29.341983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:29.344983 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:29.359609 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:29.362255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:29.392604 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1458/2903] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:33.144150 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:33.145638 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:33.160922 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8170f4-f06f-4b10-8b2f-71d1ce053883 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:33.167295 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:33.167919 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:33.168200 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:33.168470 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:33.170643 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:33.183318 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:33.184935 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:33.198811 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1446/2904] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:33.215311 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:33.226562 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008698b-5ef7-4906-b1ed-a4086c1278fa service placement] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:33.232330 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:33.233255 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:33.234000 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:33.234434 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] PATH_INFO: `/v3/users/74a63ce39dd942f9abb785d05cbd8c2b` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:33.235929 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=74a63ce39dd942f9abb785d05cbd8c2b)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:33.238373 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:33.287517 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74a63ce39dd942f9abb785d05cbd8c2b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.294553 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74a63ce39dd942f9abb785d05cbd8c2b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.295258 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74a63ce39dd942f9abb785d05cbd8c2b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.295889 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74a63ce39dd942f9abb785d05cbd8c2b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.305166 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74a63ce39dd942f9abb785d05cbd8c2b'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.323935 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1459/2905] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:55:33 2022] DELETE /identity/v3/users/74a63ce39dd942f9abb785d05cbd8c2b => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:55:33.518914 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:33.531283 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82c523-a5f9-48f5-b157-12b6366e0026 service placement] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:33.593346 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:33.593880 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:33.594155 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:33.594462 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] PATH_INFO: `/v3/projects/e6c67730fcf64392b047dd178ee72db8` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:33.595649 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c67730fcf64392b047dd178ee72db8)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:33.597568 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:33.653070 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c67730fcf64392b047dd178ee72db8', 'request_id': 'req-aeabc38a-f355-454a-99e2-6d7604b6fe19'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.659559 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c67730fcf64392b047dd178ee72db8', 'request_id': 'req-aeabc38a-f355-454a-99e2-6d7604b6fe19'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.669428 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c67730fcf64392b047dd178ee72db8', 'request_id': 'req-aeabc38a-f355-454a-99e2-6d7604b6fe19'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.678302 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:33.697637 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] The token cache is being invalidate because project e6c67730fcf64392b047dd178ee72db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:55:33.698053 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.698736 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.700780 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1447/2906] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:55:33 2022] DELETE /identity/v3/projects/e6c67730fcf64392b047dd178ee72db8 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:55:33.706203 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:33.707816 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:33.721700 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf44359-31a2-4837-9422-54c90558b58c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:33.748720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:33.786523 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:33.787381 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:33.787682 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:33.787950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] PATH_INFO: `/v3/users/336e9ea413e542aaa06487773f74bebd` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:33.789564 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=336e9ea413e542aaa06487773f74bebd)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:33.791916 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:33.844502 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '336e9ea413e542aaa06487773f74bebd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.852294 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '336e9ea413e542aaa06487773f74bebd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.853040 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '336e9ea413e542aaa06487773f74bebd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.853677 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '336e9ea413e542aaa06487773f74bebd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.864007 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '336e9ea413e542aaa06487773f74bebd'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:33.887578 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1460/2907] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:55:33 2022] DELETE /identity/v3/users/336e9ea413e542aaa06487773f74bebd => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:55:34.081624 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:34.083307 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:34.101107 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeabc38a-f355-454a-99e2-6d7604b6fe19 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:34.163800 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:34.164687 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:34.165044 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:34.165391 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] PATH_INFO: `/v3/projects/b9babb62146549a9aae03a243ecd3e67` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:34.166640 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9babb62146549a9aae03a243ecd3e67)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:34.168668 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:34.219725 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9babb62146549a9aae03a243ecd3e67', 'request_id': 'req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:34.225197 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9babb62146549a9aae03a243ecd3e67', 'request_id': 'req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:34.233578 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9babb62146549a9aae03a243ecd3e67', 'request_id': 'req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:34.245072 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] The token cache is being invalidate because project b9babb62146549a9aae03a243ecd3e67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80187) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:55:34.245365 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:34.245982 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:55:34.247545 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1448/2908] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:55:34 2022] DELETE /identity/v3/projects/b9babb62146549a9aae03a243ecd3e67 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:55:40.633049 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:55:40.634512 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:40.725972 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe804937-eef1-4108-84db-cca4d6c8bd89 admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:55:40.759379 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:40.793424 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e18ece85f014104b3c8fe92355b18a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:55:40.794039 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:55:40.794316 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:55:40.794591 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:55:40.796496 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:55:40.809342 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:55:40.810950 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:55:40.822200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1461/2909] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:55:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:55:44.242147 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:55:50.817685 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=18 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:56:39.454148 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:56:39.456008 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:56:39.531778 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f143dc8-79ad-4095-9f3e-ba4f066fd34b admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:56:39.536257 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:56:39.536928 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:56:39.537202 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:56:39.537509 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:56:39.539727 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:56:39.550709 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:56:39.552244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:56:39.560527 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1449/2910] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:56:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:56:49.556819 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=31 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:57:45.628273 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:57:45.637813 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b16911-96d8-4fc9-9d9c-2a5719424c3f service nova] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:57:45.641510 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a36d4232444f46c08f950d19ae395643', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:57:45.642079 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] REQUEST_METHOD: `GET` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:57:45.642356 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:57:45.642621 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:57:45.644451 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:57:45.708815 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:57:45.710266 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:57:45.720301 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1462/2911] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:57:51.808459 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:57:51.810427 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:57:51.827136 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82e11f-8182-44c7-9927-47e8c2fcbd20 service neutron] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:57:51.832022 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:57:51.832669 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:57:51.832976 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:57:51.833244 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] PATH_INFO: `/v3/users/d4bb35df2ffd48bf965425afcdbc42fd` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:57:51.834495 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4bb35df2ffd48bf965425afcdbc42fd)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:57:51.836639 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:57:51.892192 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4bb35df2ffd48bf965425afcdbc42fd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:51.899067 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4bb35df2ffd48bf965425afcdbc42fd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:51.908325 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4bb35df2ffd48bf965425afcdbc42fd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:51.908975 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4bb35df2ffd48bf965425afcdbc42fd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:51.909692 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4bb35df2ffd48bf965425afcdbc42fd'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:51.927013 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1450/2912] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:57:51 2022] DELETE /identity/v3/users/d4bb35df2ffd48bf965425afcdbc42fd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:57:52.109519 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:57:52.119879 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef3bbbe-a0b1-4a5e-8ebf-2da14448be3c service neutron] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:57:52.177979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:57:52.178607 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:57:52.178884 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:57:52.179190 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] PATH_INFO: `/v3/projects/a019b128831a47b7996b3d3e9d6e5652` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:57:52.180304 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a019b128831a47b7996b3d3e9d6e5652)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:57:52.182225 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:57:52.231840 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a019b128831a47b7996b3d3e9d6e5652', 'request_id': 'req-fa31f94e-2a19-489c-8863-a4c3d9b7277c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.237390 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a019b128831a47b7996b3d3e9d6e5652', 'request_id': 'req-fa31f94e-2a19-489c-8863-a4c3d9b7277c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.246213 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a019b128831a47b7996b3d3e9d6e5652', 'request_id': 'req-fa31f94e-2a19-489c-8863-a4c3d9b7277c'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.257660 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] The token cache is being invalidate because project a019b128831a47b7996b3d3e9d6e5652 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:57:52.257982 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.258720 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.260327 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1463/2913] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:57:52 2022] DELETE /identity/v3/projects/a019b128831a47b7996b3d3e9d6e5652 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:57:52.264556 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:57:52.266115 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:57:52.279729 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f09a22-27d6-4496-af09-ec494da244e7 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:57:52.335825 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:57:52.336454 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:57:52.336726 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:57:52.337001 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] PATH_INFO: `/v3/users/d3bf4100e8684c099d78e5e0eed3b4c5` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:57:52.338142 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3bf4100e8684c099d78e5e0eed3b4c5)` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:57:52.340553 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:57:52.396361 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3bf4100e8684c099d78e5e0eed3b4c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.403404 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3bf4100e8684c099d78e5e0eed3b4c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.412216 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3bf4100e8684c099d78e5e0eed3b4c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.412851 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3bf4100e8684c099d78e5e0eed3b4c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.413528 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.notifications [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3bf4100e8684c099d78e5e0eed3b4c5'} {{(pid=80187) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.421485 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:57:52.432741 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1451/2914] 10.222.0.33 () {58 vars in 1224 bytes} [Mon Aug 8 13:57:52 2022] DELETE /identity/v3/users/d3bf4100e8684c099d78e5e0eed3b4c5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:57:52.612498 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Authenticating user token {{(pid=80188) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:57:52.613947 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.fernet_utils [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80188) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:57:52.626979 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa31f94e-2a19-489c-8863-a4c3d9b7277c admin admin] Validating token access rules against request {{(pid=80188) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:57:52.652600 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:57:52.695942 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69e577935e2a4950900c965d118c5e00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'cd8972f6a1684e42be6181a89144c88c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80188) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:57:52.696628 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:57:52.696992 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:57:52.697278 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] PATH_INFO: `/v3/projects/11c8cd6d1c594e89b417c04221867541` {{(pid=80188) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:57:52.698714 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11c8cd6d1c594e89b417c04221867541)` {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:57:52.700939 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] RBAC: Authorization granted {{(pid=80188) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:57:52.748481 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11c8cd6d1c594e89b417c04221867541', 'request_id': 'req-07050216-9d40-4660-b9b6-fcd95d0276b3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.754713 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11c8cd6d1c594e89b417c04221867541', 'request_id': 'req-07050216-9d40-4660-b9b6-fcd95d0276b3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.763762 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11c8cd6d1c594e89b417c04221867541', 'request_id': 'req-07050216-9d40-4660-b9b6-fcd95d0276b3'} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.775612 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] The token cache is being invalidate because project 11c8cd6d1c594e89b417c04221867541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80188) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 08 13:57:52.775913 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.776595 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG keystone.notifications [None req-07050216-9d40-4660-b9b6-fcd95d0276b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80188) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 08 13:57:52.778200 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: [pid: 80188|app: 0|req: 1464/2915] 10.222.0.33 () {58 vars in 1233 bytes} [Mon Aug 8 13:57:52 2022] DELETE /identity/v3/projects/11c8cd6d1c594e89b417c04221867541 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 08 13:58:02.433627 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:58:02.772826 dv-cinder-851996-6-iscsi devstack@keystone.service[80188]: DEBUG dbcounter [-] [80188] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80188) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 08 13:59:35.928611 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Authenticating user token {{(pid=80187) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 08 13:59:35.930319 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:59:36.007837 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6e689-8524-4d7a-9d23-f864e9b91676 admin admin] Validating token access rules against request {{(pid=80187) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 08 13:59:36.083865 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44983e1fc3fc4079a31d1e078fb475fe', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '27dd5e0f9c7b47598842644c14231836', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80187) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 08 13:59:36.084481 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] REQUEST_METHOD: `GET` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 08 13:59:36.084753 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] SCRIPT_NAME: `/identity` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 08 13:59:36.085032 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80187) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 08 13:59:36.087373 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.fernet_utils [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80187) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 08 13:59:36.098783 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 08 13:59:36.100415 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f37cf-b586-4efe-a4ba-8ca64250bde0 service placement] RBAC: Authorization granted {{(pid=80187) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 08 13:59:36.110285 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: [pid: 80187|app: 0|req: 1452/2916] 10.222.0.33 () {60 vars in 1365 bytes} [Mon Aug 8 13:59:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 08 13:59:46.106249 dv-cinder-851996-6-iscsi devstack@keystone.service[80187]: DEBUG dbcounter [-] [80187] Writing DB stats keystone:SELECT=58 {{(pid=80187) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}